2024-09-12T01:23:03.289 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T01:23:03.310 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T01:23:03.402 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901553 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/cap-flush} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901553' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - Replacing daemon mds.a sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLt9CBzOmnphNpGK/v5LTU3I4Fzbo5tmPyG527ZzDaKzn60hy2IxSBJ2Ygs4di5r04U8vrfhFtTpInS1b6OrXow= smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCm9ElKSnb31Xz2HtSRvkQ/nVfeOWm02tjsH5mhjNuZNVvOO3JQvxMXQQDxByqajQd0iocTiwgBvENALH1R4KXg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_cap_flush teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T01:23:03.402 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T01:23:03.403 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T01:23:03.403 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T01:23:03.405 INFO:teuthology.task.internal:Checking packages... 2024-09-12T01:23:03.426 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T01:23:03.426 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T01:23:03.426 INFO:teuthology.packaging:ref: None 2024-09-12T01:23:03.426 INFO:teuthology.packaging:tag: None 2024-09-12T01:23:03.426 INFO:teuthology.packaging:branch: squid 2024-09-12T01:23:03.426 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:23:03.426 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T01:23:03.678 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T01:23:03.679 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T01:23:03.703 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T01:23:03.703 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T01:23:03.819 INFO:teuthology.task.internal:Saving configuration 2024-09-12T01:23:03.855 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T01:23:03.907 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T01:23:03.939 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901553', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:18:18.137929', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLt9CBzOmnphNpGK/v5LTU3I4Fzbo5tmPyG527ZzDaKzn60hy2IxSBJ2Ygs4di5r04U8vrfhFtTpInS1b6OrXow='} 2024-09-12T01:23:03.971 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901553', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:18:18.139598', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCm9ElKSnb31Xz2HtSRvkQ/nVfeOWm02tjsH5mhjNuZNVvOO3JQvxMXQQDxByqajQd0iocTiwgBvENALH1R4KXg='} 2024-09-12T01:23:03.971 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T01:23:03.977 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-12T01:23:03.977 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-12T01:23:03.977 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T01:23:04.053 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8cbcfbfbe0>, signals=[15]) 2024-09-12T01:23:04.053 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T01:23:04.059 INFO:teuthology.task.internal:Opening connections... 2024-09-12T01:23:04.060 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:23:04.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:23:04.149 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:23:04.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:23:04.293 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T01:23:04.299 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-09-12T01:23:04.318 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-09-12T01:23:04.318 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-09-12T01:23:04.376 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-09-12T01:23:04.376 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-09-12T01:23:04.376 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-09-12T01:23:04.376 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:23:04.376 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-09-12T01:23:04.376 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:23:04.376 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:23:04.376 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-09-12T01:23:04.376 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:23:04.377 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:23:04.377 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:23:04.377 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:23:04.377 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:23:04.377 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:23:04.378 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-09-12T01:23:04.407 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-09-12T01:23:04.426 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-09-12T01:23:04.426 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-09-12T01:23:04.483 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-09-12T01:23:04.484 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-09-12T01:23:04.484 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-09-12T01:23:04.484 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:23:04.484 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-09-12T01:23:04.484 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:23:04.484 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:23:04.484 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-09-12T01:23:04.484 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:23:04.484 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:23:04.484 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:23:04.484 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:23:04.484 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:23:04.485 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:23:04.485 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-09-12T01:23:04.515 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T01:23:04.524 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T01:23:04.529 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T01:23:04.529 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:23:04.532 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:23:04.548 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T01:23:04.554 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T01:23:04.554 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:23:04.589 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:23:04.610 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T01:23:04.677 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T01:23:04.690 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-12T01:23:04.690 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-12T01:23:04.691 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T01:23:04.691 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T01:23:04.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:23:04.708 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-09-12T01:23:04.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:23:04.709 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-12T01:23:04.765 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-503.el9.x86_64 2024-09-12T01:23:04.765 INFO:teuthology.task.kernel:Running kernel on smithi033: 5.14.0-503.el9.x86_64 2024-09-12T01:23:04.765 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum install -y kernel 2024-09-12T01:23:04.766 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-503.el9.x86_64 2024-09-12T01:23:04.767 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-503.el9.x86_64 2024-09-12T01:23:04.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-09-12T01:23:07.554 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.2 MB 00:00 2024-09-12T01:23:08.248 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-12T01:23:09.572 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-12T01:23:10.233 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-12T01:23:14.248 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.5 MB 00:00 2024-09-12T01:23:14.616 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-12T01:23:15.740 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 19 kB 00:00 2024-09-12T01:23:16.284 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 19 kB 00:00 2024-09-12T01:23:16.485 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 37 MB/s | 23 MB 00:00 2024-09-12T01:23:17.038 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-12T01:23:22.199 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T01:23:22.669 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-12T01:23:24.314 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T01:23:24.315 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T01:23:24.415 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-09-12T01:23:24.421 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-09-12T01:23:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-09-12T01:23:24.422 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-09-12T01:23:24.422 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-09-12T01:23:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T01:23:24.422 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-09-12T01:23:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T01:23:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T01:23:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T01:23:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:23:24.422 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-09-12T01:23:24.422 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-09-12T01:23:24.422 INFO:teuthology.orchestra.run.smithi033.stdout:Install 4 Packages 2024-09-12T01:23:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:23:24.423 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 79 M 2024-09-12T01:23:24.423 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 126 M 2024-09-12T01:23:24.423 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-09-12T01:23:24.559 INFO:teuthology.orchestra.run.smithi033.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 256 kB/s | 23 kB 00:00 2024-09-12T01:23:24.797 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T01:23:24.797 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T01:23:24.898 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-12T01:23:24.908 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-12T01:23:24.908 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-09-12T01:23:24.908 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-12T01:23:24.908 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-12T01:23:24.908 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T01:23:24.908 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-12T01:23:24.908 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T01:23:24.908 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T01:23:24.909 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T01:23:24.909 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:23:24.909 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-12T01:23:24.909 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-12T01:23:24.909 INFO:teuthology.orchestra.run.smithi016.stdout:Install 4 Packages 2024-09-12T01:23:24.909 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:23:24.909 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 79 M 2024-09-12T01:23:24.910 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 126 M 2024-09-12T01:23:24.910 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-12T01:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 42 kB/s | 23 kB 00:00 2024-09-12T01:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-12T01:23:26.893 INFO:teuthology.orchestra.run.smithi033.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.5 MB/s | 16 MB 00:02 2024-09-12T01:23:27.029 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-12T01:23:27.829 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 12 MB/s | 29 MB 00:02 2024-09-12T01:23:27.830 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:23:27.830 INFO:teuthology.orchestra.run.smithi016.stdout:Total 27 MB/s | 79 MB 00:02 2024-09-12T01:23:28.259 INFO:teuthology.orchestra.run.smithi033.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 7.7 MB/s | 29 MB 00:03 2024-09-12T01:23:28.545 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-12T01:23:28.931 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-12T01:23:28.931 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-12T01:23:29.310 INFO:teuthology.orchestra.run.smithi033.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 7.1 MB/s | 35 MB 00:04 2024-09-12T01:23:29.310 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:23:29.310 INFO:teuthology.orchestra.run.smithi033.stdout:Total 16 MB/s | 79 MB 00:04 2024-09-12T01:23:29.923 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-12T01:23:29.924 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-12T01:23:30.041 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-09-12T01:23:30.426 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-09-12T01:23:30.426 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-09-12T01:23:31.413 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-09-12T01:23:31.413 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-09-12T01:23:32.520 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-12T01:23:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T01:23:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:23:34.001 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-09-12T01:23:34.660 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T01:23:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:23:36.066 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:23:36.226 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:23:37.542 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:23:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:23:41.347 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:23:41.394 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:23:42.807 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:23:42.848 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:23:46.536 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:23:47.990 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:24:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:24:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:24:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:24:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T01:24:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:24:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:24:27.042 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:24:27.083 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:24:27.168 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:24:27.168 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:24:27.168 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-12T01:24:27.168 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T01:24:27.168 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T01:24:27.169 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T01:24:27.169 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T01:24:27.169 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:24:27.169 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-12T01:24:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:24:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T01:24:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:24:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:24:27.511 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T01:24:27.610 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T01:24:27.610 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T01:24:27.610 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T01:24:27.610 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T01:24:27.610 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T01:24:27.611 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-12T01:24:27.611 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T01:24:27.611 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T01:24:27.669 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T01:24:27.669 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T01:24:27.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-09-12T01:24:27.701 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-09-12T01:24:27.701 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-09-12T01:24:27.701 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-09-12T01:24:27.701 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 82.el9 2024-09-12T01:24:27.701 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-09-12T01:24:27.701 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T01:24:27.701 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-09-12T01:24:27.701 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8274670 2024-09-12T01:24:27.701 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-09-12T01:24:27.701 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-09-12T01:24:27.702 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:24:27.703 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T01:24:27.703 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T01:24:27.703 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 5.14.0-505.el9.x86_64 2024-09-12T01:24:27.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T01:24:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:24:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:24:28.302 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-09-12T01:24:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T01:24:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T01:24:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T01:24:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T01:24:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:24:28.303 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-09-12T01:24:28.362 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-12T01:24:28.647 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T01:24:28.731 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T01:24:28.731 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T01:24:28.731 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T01:24:28.731 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T01:24:28.731 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T01:24:28.731 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-12T01:24:28.731 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T01:24:28.732 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T01:24:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T01:24:28.790 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T01:24:28.790 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -qi grub2-tools 2024-09-12T01:24:28.822 INFO:teuthology.orchestra.run.smithi033.stdout:Name : grub2-tools 2024-09-12T01:24:28.822 INFO:teuthology.orchestra.run.smithi033.stdout:Epoch : 1 2024-09-12T01:24:28.822 INFO:teuthology.orchestra.run.smithi033.stdout:Version : 2.06 2024-09-12T01:24:28.822 INFO:teuthology.orchestra.run.smithi033.stdout:Release : 82.el9 2024-09-12T01:24:28.822 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: x86_64 2024-09-12T01:24:28.822 INFO:teuthology.orchestra.run.smithi033.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T01:24:28.822 INFO:teuthology.orchestra.run.smithi033.stdout:Group : Unspecified 2024-09-12T01:24:28.822 INFO:teuthology.orchestra.run.smithi033.stdout:Size : 8274670 2024-09-12T01:24:28.822 INFO:teuthology.orchestra.run.smithi033.stdout:License : GPLv3+ 2024-09-12T01:24:28.822 INFO:teuthology.orchestra.run.smithi033.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout:Packager : builder@centos.org 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout:Vendor : CentOS 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout:Summary : Support tools for GRUB. 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout:Description : 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout:hardware devices. 2024-09-12T01:24:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:24:28.824 INFO:teuthology.orchestra.run.smithi033.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T01:24:28.824 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T01:24:28.824 INFO:teuthology.task.kernel:Updating grub on smithi033 to boot 5.14.0-505.el9.x86_64 2024-09-12T01:24:28.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T01:24:29.404 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2024-09-12T01:24:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T01:24:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-12T01:24:29.900 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-12T01:24:29.915 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.EiP2Pu3KuG 2024-09-12T01:24:29.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EiP2Pu3KuG 2024-09-12T01:24:29.981 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.EiP2Pu3KuG 2024-09-12T01:24:30.121 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.EiP2Pu3KuG is 6KB 2024-09-12T01:24:30.170 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.EiP2Pu3KuG 2024-09-12T01:24:30.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T01:24:30.246 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T01:24:30.246 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T01:24:30.246 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T01:24:30.246 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T01:24:30.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T01:24:30.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-09-12T01:24:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T01:24:30.695 INFO:teuthology.orchestra.run.smithi033.stderr:done 2024-09-12T01:24:30.697 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-09-12T01:24:30.712 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.vcAq8PSjmV 2024-09-12T01:24:30.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.vcAq8PSjmV 2024-09-12T01:24:30.778 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.vcAq8PSjmV 2024-09-12T01:24:30.923 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.vcAq8PSjmV is 6KB 2024-09-12T01:24:30.972 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.vcAq8PSjmV 2024-09-12T01:24:30.987 DEBUG:teuthology.orchestra.run.smithi033:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T01:24:31.050 INFO:teuthology.orchestra.run.smithi033.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T01:24:31.050 INFO:teuthology.orchestra.run.smithi033.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T01:24:31.051 INFO:teuthology.orchestra.run.smithi033.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T01:24:31.051 INFO:teuthology.orchestra.run.smithi033.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T01:24:31.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T01:24:31.217 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2024-09-12T01:25:00.549 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-12T01:25:00.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T01:25:00.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:25:01.220 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-12T01:25:01.220 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-12T01:25:01.221 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:25:19.037 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-09-12T01:25:19.549 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-09-12T01:25:28.043 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T01:25:28.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:25:28.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-12T01:25:28.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:25:46.941 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-09-12T01:25:58.953 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-12T01:25:58.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:25:59.708 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-09-12T01:26:00.116 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-12T01:26:00.117 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-12T01:26:00.117 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-09-12T01:26:00.132 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-505.el9.x86_64 2024-09-12T01:26:00.132 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T01:26:00.132 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T01:26:00.133 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T01:26:01.133 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-12T01:26:01.133 DEBUG:teuthology.orchestra.run.smithi033:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T01:26:01.237 INFO:teuthology.orchestra.run.smithi033.stdout:ttyS1 2024-09-12T01:26:01.258 DEBUG:teuthology.parallel:result is None 2024-09-12T01:26:28.055 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T01:26:40.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T01:26:40.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:26:40.457 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-12T01:26:40.836 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T01:26:40.836 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-12T01:26:40.837 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-12T01:26:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-505.el9.x86_64 2024-09-12T01:26:40.854 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T01:26:40.854 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T01:26:40.854 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T01:26:41.855 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-12T01:26:41.855 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T01:26:41.946 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2024-09-12T01:26:41.967 DEBUG:teuthology.parallel:result is None 2024-09-12T01:26:41.967 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T01:26:41.974 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T01:26:41.974 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T01:26:41.977 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T01:26:41.999 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T01:26:42.005 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T01:26:42.011 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T01:26:42.011 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T01:26:42.037 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T01:26:42.070 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T01:26:42.076 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T01:26:42.076 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T01:26:42.098 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T01:26:42.135 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:26:42.137 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:26:42.149 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:26:42.150 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:26:42.151 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T01:26:42.157 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T01:26:42.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T01:26:42.193 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T01:26:42.221 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T01:26:42.231 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T01:26:42.231 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T01:26:42.261 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T01:26:42.278 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:26:42.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:26:42.413 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:26:42.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:26:42.541 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:26:42.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T01:26:42.610 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:26:42.664 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:26:42.714 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:26:42.807 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:26:42.843 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:26:42.843 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T01:26:42.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-09-12T01:26:42.915 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-09-12T01:26:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:26:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:26:43.420 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T01:26:43.427 INFO:teuthology.task.internal:Starting timer... 2024-09-12T01:26:43.427 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T01:26:43.436 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T01:26:43.444 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-12T01:26:43.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-09-12T01:26:43.507 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-09-12T01:26:43.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-09-12T01:26:43.603 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-09-12T01:26:43.605 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T01:26:43.606 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-09-12T01:26:43.642 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-09-12T01:26:43.643 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-09-12T01:26:43.667 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-09-12T01:26:43.668 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi033.front.sepia.ceph.com': 'permissive'} 2024-09-12T01:26:43.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:26:43.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:26:43.711 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:26:43.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:26:43.740 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T01:26:43.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-09-12T01:26:43.787 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-09-12T01:26:43.823 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T01:26:43.832 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-12T01:26:43.836 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-12T01:26:43.969 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T01:26:43.995 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T01:26:43.996 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi033.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T01:30:34.383 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi033.front.sepia.ceph.com')] 2024-09-12T01:30:34.385 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T01:30:34.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:30:34.475 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-12T01:30:34.566 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T01:30:34.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-12T01:30:34.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:30:34.656 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-09-12T01:30:34.739 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-12T01:30:34.739 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T01:30:34.749 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T01:30:34.749 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T01:30:34.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:30:34.752 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T01:30:34.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:30:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T01:30:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T01:30:34.812 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T01:30:34.824 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-09-12T01:30:34.827 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T01:30:34.836 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-09-12T01:30:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T01:30:34.854 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-09-12T01:30:34.866 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-09-12T01:30:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T01:30:34.881 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T01:30:34.896 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T01:30:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:30:34.930 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:30:34.930 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-12T01:30:34.931 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:30:34.931 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:30:34.931 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:30:34.931 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:30:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:30:34.959 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:30:34.960 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-09-12T01:30:34.960 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:30:34.961 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:30:34.961 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:30:34.961 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:30:34.962 INFO:teuthology.run_tasks:Running task install... 2024-09-12T01:30:34.969 DEBUG:teuthology.task.install:project ceph 2024-09-12T01:30:34.969 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T01:30:34.970 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T01:30:34.970 INFO:teuthology.task.install:Using flavor: default 2024-09-12T01:30:34.975 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T01:30:34.975 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T01:30:34.976 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T01:30:34.976 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:30:34.979 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T01:30:34.979 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:30:35.136 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T01:30:35.136 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T01:30:35.152 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T01:30:35.152 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T01:30:35.262 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T01:30:35.263 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:30:35.263 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T01:30:35.274 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T01:30:35.274 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:30:35.274 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T01:30:35.290 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T01:30:35.291 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T01:30:35.302 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T01:30:35.303 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T01:30:35.358 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T01:30:35.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T01:30:35.436 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T01:30:35.450 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T01:30:35.502 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-09-12T01:30:35.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-09-12T01:30:35.516 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-09-12T01:30:35.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-09-12T01:30:35.755 INFO:teuthology.orchestra.run.smithi033.stdout:45 files removed 2024-09-12T01:30:35.756 INFO:teuthology.orchestra.run.smithi016.stdout:45 files removed 2024-09-12T01:30:35.783 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T01:30:35.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T01:30:36.364 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-12T01:30:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 294 kB/s | 84 kB 00:00 2024-09-12T01:30:36.648 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T01:30:36.661 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-12T01:30:36.934 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-12T01:30:36.953 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 7.5 kB/s | 1.9 kB 00:00 2024-09-12T01:30:37.495 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-12T01:30:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.2 MB 00:01 2024-09-12T01:30:39.463 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-12T01:30:39.917 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-12T01:30:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.5 MB 00:00 2024-09-12T01:30:44.351 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-12T01:30:45.798 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 104 kB/s | 19 kB 00:00 2024-09-12T01:30:46.094 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-12T01:30:46.885 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-12T01:30:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-12T01:30:52.516 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T01:30:52.695 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T01:30:55.002 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:30:55.003 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:30:55.014 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T01:30:55.015 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T01:30:55.016 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T01:30:55.017 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T01:30:55.018 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T01:30:55.021 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T01:30:55.022 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T01:30:55.088 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-09-12T01:30:55.117 INFO:teuthology.orchestra.run.smithi033.stdout:====================================================================================== 2024-09-12T01:30:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-09-12T01:30:55.117 INFO:teuthology.orchestra.run.smithi033.stdout:====================================================================================== 2024-09-12T01:30:55.117 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-09-12T01:30:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T01:30:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T01:30:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T01:30:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T01:30:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T01:30:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T01:30:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T01:30:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T01:30:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T01:30:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T01:30:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T01:30:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T01:30:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T01:30:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T01:30:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T01:30:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T01:30:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T01:30:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T01:30:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T01:30:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T01:30:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T01:30:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T01:30:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T01:30:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T01:30:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T01:30:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T01:30:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T01:30:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T01:30:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T01:30:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T01:30:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T01:30:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T01:30:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T01:30:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T01:30:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T01:30:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T01:30:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T01:30:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T01:30:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T01:30:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T01:30:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T01:30:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T01:30:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T01:30:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T01:30:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T01:30:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T01:30:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T01:30:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T01:30:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T01:30:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T01:30:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T01:30:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T01:30:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T01:30:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T01:30:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T01:30:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T01:30:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T01:30:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T01:30:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T01:30:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T01:30:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T01:30:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T01:30:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T01:30:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T01:30:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T01:30:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T01:30:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T01:30:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T01:30:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T01:30:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T01:30:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T01:30:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T01:30:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T01:30:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T01:30:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T01:30:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T01:30:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T01:30:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T01:30:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T01:30:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T01:30:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T01:30:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T01:30:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T01:30:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T01:30:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T01:30:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T01:30:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T01:30:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T01:30:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T01:30:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T01:30:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T01:30:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T01:30:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T01:30:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T01:30:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T01:30:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T01:30:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T01:30:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T01:30:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T01:30:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T01:30:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T01:30:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T01:30:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T01:30:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T01:30:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T01:30:55.129 INFO:teuthology.orchestra.run.smithi033.stdout:Installing weak dependencies: 2024-09-12T01:30:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T01:30:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:30:55.130 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-09-12T01:30:55.130 INFO:teuthology.orchestra.run.smithi033.stdout:====================================================================================== 2024-09-12T01:30:55.130 INFO:teuthology.orchestra.run.smithi033.stdout:Install 148 Packages 2024-09-12T01:30:55.130 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 8 Packages 2024-09-12T01:30:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:30:55.130 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 226 M 2024-09-12T01:30:55.131 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-09-12T01:30:55.132 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:30:55.132 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:30:55.144 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T01:30:55.145 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T01:30:55.146 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T01:30:55.147 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T01:30:55.148 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T01:30:55.151 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T01:30:55.151 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T01:30:55.220 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-12T01:30:55.236 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-12T01:30:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-09-12T01:30:55.236 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-12T01:30:55.236 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-12T01:30:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T01:30:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T01:30:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T01:30:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T01:30:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T01:30:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T01:30:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T01:30:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T01:30:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T01:30:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T01:30:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T01:30:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T01:30:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T01:30:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T01:30:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T01:30:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T01:30:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T01:30:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T01:30:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T01:30:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T01:30:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T01:30:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T01:30:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T01:30:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T01:30:55.239 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-09-12T01:30:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T01:30:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T01:30:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T01:30:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T01:30:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T01:30:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T01:30:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T01:30:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T01:30:55.240 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-12T01:30:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T01:30:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T01:30:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T01:30:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T01:30:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T01:30:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T01:30:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T01:30:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T01:30:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T01:30:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T01:30:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T01:30:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T01:30:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T01:30:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T01:30:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T01:30:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T01:30:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T01:30:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T01:30:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T01:30:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T01:30:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T01:30:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T01:30:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T01:30:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T01:30:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T01:30:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T01:30:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T01:30:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T01:30:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T01:30:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T01:30:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T01:30:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T01:30:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T01:30:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T01:30:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T01:30:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T01:30:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T01:30:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T01:30:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T01:30:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T01:30:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T01:30:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T01:30:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T01:30:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T01:30:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T01:30:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T01:30:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T01:30:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T01:30:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T01:30:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T01:30:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T01:30:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T01:30:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T01:30:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T01:30:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T01:30:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T01:30:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T01:30:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T01:30:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T01:30:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T01:30:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T01:30:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T01:30:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T01:30:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T01:30:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T01:30:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T01:30:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T01:30:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T01:30:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T01:30:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T01:30:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T01:30:55.249 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-09-12T01:30:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T01:30:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:30:55.249 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-12T01:30:55.249 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-12T01:30:55.249 INFO:teuthology.orchestra.run.smithi016.stdout:Install 148 Packages 2024-09-12T01:30:55.249 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 8 Packages 2024-09-12T01:30:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:30:55.250 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 226 M 2024-09-12T01:30:55.250 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-12T01:30:55.826 INFO:teuthology.orchestra.run.smithi033.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-12T01:30:56.009 INFO:teuthology.orchestra.run.smithi033.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 4.8 MB/s | 897 kB 00:00 2024-09-12T01:30:56.065 INFO:teuthology.orchestra.run.smithi016.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 37 kB/s | 6.5 kB 00:00 2024-09-12T01:30:56.101 INFO:teuthology.orchestra.run.smithi033.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.5 MB/s | 145 kB 00:00 2024-09-12T01:30:56.210 INFO:teuthology.orchestra.run.smithi033.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-12T01:30:56.285 INFO:teuthology.orchestra.run.smithi033.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 2.4 MB 00:00 2024-09-12T01:30:56.335 INFO:teuthology.orchestra.run.smithi033.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 1.5 MB 00:00 2024-09-12T01:30:56.707 INFO:teuthology.orchestra.run.smithi016.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.4 MB/s | 897 kB 00:00 2024-09-12T01:30:56.727 INFO:teuthology.orchestra.run.smithi033.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-12T01:30:56.809 INFO:teuthology.orchestra.run.smithi016.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.4 MB/s | 145 kB 00:00 2024-09-12T01:30:56.944 INFO:teuthology.orchestra.run.smithi033.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 17 MB/s | 21 MB 00:01 2024-09-12T01:30:57.003 INFO:teuthology.orchestra.run.smithi033.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 434 kB/s | 25 kB 00:00 2024-09-12T01:30:57.383 INFO:teuthology.orchestra.run.smithi016.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 3.6 MB/s | 5.4 MB 00:01 2024-09-12T01:30:57.450 INFO:teuthology.orchestra.run.smithi016.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.7 MB/s | 2.4 MB 00:00 2024-09-12T01:30:57.461 INFO:teuthology.orchestra.run.smithi033.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 15 MB/s | 17 MB 00:01 2024-09-12T01:30:57.583 INFO:teuthology.orchestra.run.smithi016.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 7.6 MB/s | 1.5 MB 00:00 2024-09-12T01:30:57.595 INFO:teuthology.orchestra.run.smithi033.stdout:(11/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 12 MB/s | 11 MB 00:00 2024-09-12T01:30:57.620 INFO:teuthology.orchestra.run.smithi033.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.4 MB/s | 228 kB 00:00 2024-09-12T01:30:57.654 INFO:teuthology.orchestra.run.smithi033.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 463 kB/s | 27 kB 00:00 2024-09-12T01:30:57.679 INFO:teuthology.orchestra.run.smithi033.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 544 kB/s | 32 kB 00:00 2024-09-12T01:30:57.721 INFO:teuthology.orchestra.run.smithi033.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 752 kB 00:00 2024-09-12T01:30:57.746 INFO:teuthology.orchestra.run.smithi033.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 163 kB 00:00 2024-09-12T01:30:57.797 INFO:teuthology.orchestra.run.smithi033.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 127 kB 00:00 2024-09-12T01:30:57.838 INFO:teuthology.orchestra.run.smithi033.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.4 MB/s | 506 kB 00:00 2024-09-12T01:30:57.939 INFO:teuthology.orchestra.run.smithi033.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 452 kB/s | 45 kB 00:00 2024-09-12T01:30:58.039 INFO:teuthology.orchestra.run.smithi033.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.3 MB/s | 136 kB 00:00 2024-09-12T01:30:58.148 INFO:teuthology.orchestra.run.smithi033.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.5 MB/s | 162 kB 00:00 2024-09-12T01:30:58.250 INFO:teuthology.orchestra.run.smithi016.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 5.9 MB/s | 4.7 MB 00:00 2024-09-12T01:30:58.256 INFO:teuthology.orchestra.run.smithi033.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 2.9 MB/s | 321 kB 00:00 2024-09-12T01:30:58.390 INFO:teuthology.orchestra.run.smithi033.stdout:(23/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 9.0 MB/s | 5.3 MB 00:00 2024-09-12T01:30:58.415 INFO:teuthology.orchestra.run.smithi033.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 301 kB 00:00 2024-09-12T01:30:58.482 INFO:teuthology.orchestra.run.smithi033.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.1 MB/s | 100 kB 00:00 2024-09-12T01:30:58.516 INFO:teuthology.orchestra.run.smithi033.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 850 kB/s | 85 kB 00:00 2024-09-12T01:30:58.958 INFO:teuthology.orchestra.run.smithi033.stdout:(27/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 6.5 MB/s | 3.1 MB 00:00 2024-09-12T01:30:59.041 INFO:teuthology.orchestra.run.smithi033.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 383 kB/s | 32 kB 00:00 2024-09-12T01:30:59.092 INFO:teuthology.orchestra.run.smithi033.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.9 MB/s | 146 kB 00:00 2024-09-12T01:30:59.184 INFO:teuthology.orchestra.run.smithi016.stdout:(8/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 17 MB 00:01 2024-09-12T01:30:59.234 INFO:teuthology.orchestra.run.smithi016.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 506 kB/s | 25 kB 00:00 2024-09-12T01:30:59.468 INFO:teuthology.orchestra.run.smithi016.stdout:(10/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 5.9 MB/s | 21 MB 00:03 2024-09-12T01:30:59.526 INFO:teuthology.orchestra.run.smithi016.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 228 kB 00:00 2024-09-12T01:30:59.567 INFO:teuthology.orchestra.run.smithi033.stdout:(30/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 19 MB/s | 49 MB 00:02 2024-09-12T01:30:59.592 INFO:teuthology.orchestra.run.smithi033.stdout:(31/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 159 kB/s | 171 kB 00:01 2024-09-12T01:30:59.668 INFO:teuthology.orchestra.run.smithi016.stdout:(12/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 7.5 MB/s | 11 MB 00:01 2024-09-12T01:30:59.676 INFO:teuthology.orchestra.run.smithi033.stdout:(32/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 6.1 MB/s | 3.6 MB 00:00 2024-09-12T01:30:59.685 INFO:teuthology.orchestra.run.smithi016.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 170 kB/s | 27 kB 00:00 2024-09-12T01:30:59.702 INFO:teuthology.orchestra.run.smithi016.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 946 kB/s | 32 kB 00:00 2024-09-12T01:30:59.719 INFO:teuthology.orchestra.run.smithi033.stdout:(33/156): ceph-mgr-modules-core-19.1.1-298.gb53 1.9 MB/s | 249 kB 00:00 2024-09-12T01:30:59.752 INFO:teuthology.orchestra.run.smithi016.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 3.2 MB/s | 163 kB 00:00 2024-09-12T01:30:59.778 INFO:teuthology.orchestra.run.smithi016.stdout:(16/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.0 MB/s | 752 kB 00:00 2024-09-12T01:30:59.795 INFO:teuthology.orchestra.run.smithi016.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 3.0 MB/s | 127 kB 00:00 2024-09-12T01:30:59.837 INFO:teuthology.orchestra.run.smithi016.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 8.4 MB/s | 506 kB 00:00 2024-09-12T01:30:59.868 INFO:teuthology.orchestra.run.smithi033.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 257 kB/s | 49 kB 00:00 2024-09-12T01:30:59.887 INFO:teuthology.orchestra.run.smithi016.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 904 kB/s | 45 kB 00:00 2024-09-12T01:30:59.893 INFO:teuthology.orchestra.run.smithi033.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 94 kB/s | 16 kB 00:00 2024-09-12T01:30:59.927 INFO:teuthology.orchestra.run.smithi033.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-12T01:30:59.945 INFO:teuthology.orchestra.run.smithi016.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.3 MB/s | 136 kB 00:00 2024-09-12T01:30:59.994 INFO:teuthology.orchestra.run.smithi033.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 274 kB/s | 18 kB 00:00 2024-09-12T01:31:00.012 INFO:teuthology.orchestra.run.smithi016.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-12T01:31:00.028 INFO:teuthology.orchestra.run.smithi033.stdout:(38/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 5.6 MB/s | 765 kB 00:00 2024-09-12T01:31:00.080 INFO:teuthology.orchestra.run.smithi016.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 4.7 MB/s | 321 kB 00:00 2024-09-12T01:31:00.138 INFO:teuthology.orchestra.run.smithi016.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.1 MB/s | 301 kB 00:00 2024-09-12T01:31:00.153 INFO:teuthology.orchestra.run.smithi033.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 13 MB/s | 7.4 MB 00:00 2024-09-12T01:31:00.178 INFO:teuthology.orchestra.run.smithi033.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 434 kB/s | 80 kB 00:00 2024-09-12T01:31:00.196 INFO:teuthology.orchestra.run.smithi016.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-12T01:31:00.262 INFO:teuthology.orchestra.run.smithi033.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.3 MB/s | 801 kB 00:00 2024-09-12T01:31:00.280 INFO:teuthology.orchestra.run.smithi016.stdout:(25/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 5.3 MB 00:00 2024-09-12T01:31:00.287 INFO:teuthology.orchestra.run.smithi033.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-12T01:31:00.306 INFO:teuthology.orchestra.run.smithi016.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 784 kB/s | 85 kB 00:00 2024-09-12T01:31:00.312 INFO:teuthology.orchestra.run.smithi033.stdout:(43/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-12T01:31:00.338 INFO:teuthology.orchestra.run.smithi033.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-12T01:31:00.363 INFO:teuthology.orchestra.run.smithi033.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-12T01:31:00.364 INFO:teuthology.orchestra.run.smithi016.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-12T01:31:00.388 INFO:teuthology.orchestra.run.smithi033.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T01:31:00.422 INFO:teuthology.orchestra.run.smithi033.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-12T01:31:00.423 INFO:teuthology.orchestra.run.smithi016.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 547 kB/s | 32 kB 00:00 2024-09-12T01:31:00.481 INFO:teuthology.orchestra.run.smithi016.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.4 MB/s | 146 kB 00:00 2024-09-12T01:31:00.539 INFO:teuthology.orchestra.run.smithi033.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 4.4 MB/s | 1.2 MB 00:00 2024-09-12T01:31:00.541 INFO:teuthology.orchestra.run.smithi016.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-12T01:31:00.664 INFO:teuthology.orchestra.run.smithi033.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-12T01:31:00.745 INFO:teuthology.orchestra.run.smithi033.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 270 kB/s | 22 kB 00:00 2024-09-12T01:31:00.749 INFO:teuthology.orchestra.run.smithi016.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 13 MB/s | 3.6 MB 00:00 2024-09-12T01:31:00.765 INFO:teuthology.orchestra.run.smithi033.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 473 kB/s | 107 kB 00:00 2024-09-12T01:31:00.799 INFO:teuthology.orchestra.run.smithi016.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T01:31:00.807 INFO:teuthology.orchestra.run.smithi033.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.0 MB/s | 313 kB 00:00 2024-09-12T01:31:00.832 INFO:teuthology.orchestra.run.smithi033.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-09-12T01:31:00.857 INFO:teuthology.orchestra.run.smithi033.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 2.1 MB/s | 943 kB 00:00 2024-09-12T01:31:00.858 INFO:teuthology.orchestra.run.smithi016.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 845 kB/s | 49 kB 00:00 2024-09-12T01:31:00.882 INFO:teuthology.orchestra.run.smithi033.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-12T01:31:00.916 INFO:teuthology.orchestra.run.smithi033.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-09-12T01:31:00.917 INFO:teuthology.orchestra.run.smithi016.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 283 kB/s | 16 kB 00:00 2024-09-12T01:31:00.941 INFO:teuthology.orchestra.run.smithi033.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-12T01:31:00.975 INFO:teuthology.orchestra.run.smithi033.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 774 kB/s | 45 kB 00:00 2024-09-12T01:31:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.2 MB/s | 285 kB 00:00 2024-09-12T01:31:01.084 INFO:teuthology.orchestra.run.smithi033.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-12T01:31:01.117 INFO:teuthology.orchestra.run.smithi033.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-09-12T01:31:01.142 INFO:teuthology.orchestra.run.smithi016.stdout:(36/156): ceph-mgr-diskprediction-local-19.1.1- 12 MB/s | 7.4 MB 00:00 2024-09-12T01:31:01.143 INFO:teuthology.orchestra.run.smithi033.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-12T01:31:01.167 INFO:teuthology.orchestra.run.smithi016.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 4.1 MB/s | 765 kB 00:00 2024-09-12T01:31:01.168 INFO:teuthology.orchestra.run.smithi033.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 2.9 MB/s | 581 kB 00:00 2024-09-12T01:31:01.193 INFO:teuthology.orchestra.run.smithi033.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-12T01:31:01.201 INFO:teuthology.orchestra.run.smithi016.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 311 kB/s | 18 kB 00:00 2024-09-12T01:31:01.227 INFO:teuthology.orchestra.run.smithi033.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-12T01:31:01.252 INFO:teuthology.orchestra.run.smithi033.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-12T01:31:01.277 INFO:teuthology.orchestra.run.smithi033.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 459 kB/s | 38 kB 00:00 2024-09-12T01:31:01.335 INFO:teuthology.orchestra.run.smithi016.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-12T01:31:01.569 INFO:teuthology.orchestra.run.smithi033.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 14 MB/s | 4.9 MB 00:00 2024-09-12T01:31:01.611 INFO:teuthology.orchestra.run.smithi033.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 4.8 MB/s | 1.6 MB 00:00 2024-09-12T01:31:01.678 INFO:teuthology.orchestra.run.smithi033.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-12T01:31:01.703 INFO:teuthology.orchestra.run.smithi033.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-12T01:31:01.737 INFO:teuthology.orchestra.run.smithi033.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-12T01:31:01.795 INFO:teuthology.orchestra.run.smithi033.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T01:31:01.810 INFO:teuthology.orchestra.run.smithi016.stdout:(40/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 19 MB/s | 49 MB 00:02 2024-09-12T01:31:01.835 INFO:teuthology.orchestra.run.smithi016.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 382 kB/s | 191 kB 00:00 2024-09-12T01:31:01.854 INFO:teuthology.orchestra.run.smithi033.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-09-12T01:31:01.946 INFO:teuthology.orchestra.run.smithi033.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 668 kB/s | 61 kB 00:00 2024-09-12T01:31:01.960 INFO:teuthology.orchestra.run.smithi016.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 801 kB 00:00 2024-09-12T01:31:02.019 INFO:teuthology.orchestra.run.smithi016.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-12T01:31:02.021 INFO:teuthology.orchestra.run.smithi033.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 3.2 MB/s | 245 kB 00:00 2024-09-12T01:31:02.052 INFO:teuthology.orchestra.run.smithi016.stdout:(44/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-12T01:31:02.078 INFO:teuthology.orchestra.run.smithi016.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-12T01:31:02.097 INFO:teuthology.orchestra.run.smithi033.stdout:(76/156): postgresql-server-13.16-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-09-12T01:31:02.130 INFO:teuthology.orchestra.run.smithi033.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-12T01:31:02.136 INFO:teuthology.orchestra.run.smithi016.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-12T01:31:02.155 INFO:teuthology.orchestra.run.smithi033.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-09-12T01:31:02.161 INFO:teuthology.orchestra.run.smithi016.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-12T01:31:02.204 INFO:teuthology.orchestra.run.smithi016.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 3.3 MB/s | 1.2 MB 00:00 2024-09-12T01:31:02.237 INFO:teuthology.orchestra.run.smithi016.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 6.7 MB/s | 689 kB 00:00 2024-09-12T01:31:02.262 INFO:teuthology.orchestra.run.smithi016.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-12T01:31:02.264 INFO:teuthology.orchestra.run.smithi033.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-12T01:31:02.288 INFO:teuthology.orchestra.run.smithi016.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-12T01:31:02.290 INFO:teuthology.orchestra.run.smithi033.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 120 kB/s | 19 kB 00:00 2024-09-12T01:31:02.323 INFO:teuthology.orchestra.run.smithi033.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-12T01:31:02.346 INFO:teuthology.orchestra.run.smithi016.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 561 kB/s | 33 kB 00:00 2024-09-12T01:31:02.348 INFO:teuthology.orchestra.run.smithi033.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 416 kB/s | 35 kB 00:00 2024-09-12T01:31:02.380 INFO:teuthology.orchestra.run.smithi016.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-12T01:31:02.382 INFO:teuthology.orchestra.run.smithi033.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-12T01:31:02.407 INFO:teuthology.orchestra.run.smithi033.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-12T01:31:02.439 INFO:teuthology.orchestra.run.smithi016.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-09-12T01:31:02.441 INFO:teuthology.orchestra.run.smithi033.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-12T01:31:02.464 INFO:teuthology.orchestra.run.smithi016.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-09-12T01:31:02.466 INFO:teuthology.orchestra.run.smithi033.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-09-12T01:31:02.500 INFO:teuthology.orchestra.run.smithi033.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-12T01:31:02.525 INFO:teuthology.orchestra.run.smithi033.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-12T01:31:02.537 INFO:teuthology.orchestra.run.smithi016.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 153 kB/s | 11 kB 00:00 2024-09-12T01:31:02.559 INFO:teuthology.orchestra.run.smithi033.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-12T01:31:02.576 INFO:teuthology.orchestra.run.smithi033.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-12T01:31:02.601 INFO:teuthology.orchestra.run.smithi033.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 54 kB 00:00 2024-09-12T01:31:02.643 INFO:teuthology.orchestra.run.smithi033.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-12T01:31:02.693 INFO:teuthology.orchestra.run.smithi033.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-12T01:31:02.706 INFO:teuthology.orchestra.run.smithi016.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 663 kB/s | 160 kB 00:00 2024-09-12T01:31:02.740 INFO:teuthology.orchestra.run.smithi016.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 223 kB/s | 45 kB 00:00 2024-09-12T01:31:02.768 INFO:teuthology.orchestra.run.smithi033.stdout:(94/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-12T01:31:02.794 INFO:teuthology.orchestra.run.smithi033.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-12T01:31:02.807 INFO:teuthology.orchestra.run.smithi016.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.5 MB/s | 3.0 MB 00:00 2024-09-12T01:31:02.819 INFO:teuthology.orchestra.run.smithi033.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-12T01:31:02.844 INFO:teuthology.orchestra.run.smithi033.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-12T01:31:02.865 INFO:teuthology.orchestra.run.smithi016.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 420 kB/s | 24 kB 00:00 2024-09-12T01:31:02.878 INFO:teuthology.orchestra.run.smithi033.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-12T01:31:02.891 INFO:teuthology.orchestra.run.smithi016.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-09-12T01:31:02.903 INFO:teuthology.orchestra.run.smithi033.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-09-12T01:31:02.916 INFO:teuthology.orchestra.run.smithi016.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-12T01:31:02.936 INFO:teuthology.orchestra.run.smithi033.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-12T01:31:02.950 INFO:teuthology.orchestra.run.smithi016.stdout:(63/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.7 MB/s | 662 kB 00:00 2024-09-12T01:31:02.962 INFO:teuthology.orchestra.run.smithi033.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-12T01:31:02.975 INFO:teuthology.orchestra.run.smithi016.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-12T01:31:02.987 INFO:teuthology.orchestra.run.smithi033.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-12T01:31:03.000 INFO:teuthology.orchestra.run.smithi016.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-12T01:31:03.025 INFO:teuthology.orchestra.run.smithi016.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-12T01:31:03.054 INFO:teuthology.orchestra.run.smithi033.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-09-12T01:31:03.246 INFO:teuthology.orchestra.run.smithi033.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-09-12T01:31:03.288 INFO:teuthology.orchestra.run.smithi033.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-09-12T01:31:03.313 INFO:teuthology.orchestra.run.smithi033.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 858 kB/s | 19 kB 00:00 2024-09-12T01:31:03.317 INFO:teuthology.orchestra.run.smithi016.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-12T01:31:03.349 INFO:teuthology.orchestra.run.smithi033.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-09-12T01:31:03.359 INFO:teuthology.orchestra.run.smithi016.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 4.8 MB/s | 1.6 MB 00:00 2024-09-12T01:31:03.381 INFO:teuthology.orchestra.run.smithi033.stdout:(108/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.3 MB/s | 309 kB 00:00 2024-09-12T01:31:03.414 INFO:teuthology.orchestra.run.smithi033.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2024-09-12T01:31:03.418 INFO:teuthology.orchestra.run.smithi016.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-12T01:31:03.439 INFO:teuthology.orchestra.run.smithi033.stdout:(110/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 281 kB/s | 26 kB 00:00 2024-09-12T01:31:03.443 INFO:teuthology.orchestra.run.smithi016.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-12T01:31:03.465 INFO:teuthology.orchestra.run.smithi033.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-12T01:31:03.490 INFO:teuthology.orchestra.run.smithi033.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-12T01:31:03.549 INFO:teuthology.orchestra.run.smithi033.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-12T01:31:03.574 INFO:teuthology.orchestra.run.smithi033.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T01:31:03.576 INFO:teuthology.orchestra.run.smithi016.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 8.2 MB/s | 4.9 MB 00:00 2024-09-12T01:31:03.599 INFO:teuthology.orchestra.run.smithi033.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T01:31:03.625 INFO:teuthology.orchestra.run.smithi033.stdout:(116/156): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-09-12T01:31:03.635 INFO:teuthology.orchestra.run.smithi016.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T01:31:03.650 INFO:teuthology.orchestra.run.smithi033.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-12T01:31:03.669 INFO:teuthology.orchestra.run.smithi016.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-09-12T01:31:03.683 INFO:teuthology.orchestra.run.smithi033.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-12T01:31:03.717 INFO:teuthology.orchestra.run.smithi033.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-12T01:31:03.751 INFO:teuthology.orchestra.run.smithi033.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-12T01:31:03.769 INFO:teuthology.orchestra.run.smithi016.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-09-12T01:31:03.776 INFO:teuthology.orchestra.run.smithi033.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-12T01:31:03.794 INFO:teuthology.orchestra.run.smithi016.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 489 kB/s | 61 kB 00:00 2024-09-12T01:31:03.801 INFO:teuthology.orchestra.run.smithi033.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-12T01:31:03.827 INFO:teuthology.orchestra.run.smithi033.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-12T01:31:03.828 INFO:teuthology.orchestra.run.smithi016.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-12T01:31:03.852 INFO:teuthology.orchestra.run.smithi033.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 714 kB/s | 18 kB 00:00 2024-09-12T01:31:03.861 INFO:teuthology.orchestra.run.smithi016.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-09-12T01:31:03.877 INFO:teuthology.orchestra.run.smithi033.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-12T01:31:03.887 INFO:teuthology.orchestra.run.smithi016.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-12T01:31:03.912 INFO:teuthology.orchestra.run.smithi016.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-12T01:31:03.946 INFO:teuthology.orchestra.run.smithi016.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-12T01:31:03.952 INFO:teuthology.orchestra.run.smithi033.stdout:(126/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-09-12T01:31:03.971 INFO:teuthology.orchestra.run.smithi016.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-09-12T01:31:03.977 INFO:teuthology.orchestra.run.smithi033.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-09-12T01:31:04.003 INFO:teuthology.orchestra.run.smithi033.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-12T01:31:04.028 INFO:teuthology.orchestra.run.smithi033.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T01:31:04.030 INFO:teuthology.orchestra.run.smithi016.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-12T01:31:04.053 INFO:teuthology.orchestra.run.smithi033.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T01:31:04.079 INFO:teuthology.orchestra.run.smithi033.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T01:31:04.112 INFO:teuthology.orchestra.run.smithi033.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-12T01:31:04.138 INFO:teuthology.orchestra.run.smithi033.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-12T01:31:04.147 INFO:teuthology.orchestra.run.smithi016.stdout:(83/156): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-12T01:31:04.172 INFO:teuthology.orchestra.run.smithi016.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-09-12T01:31:04.180 INFO:teuthology.orchestra.run.smithi033.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-09-12T01:31:04.205 INFO:teuthology.orchestra.run.smithi033.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-12T01:31:04.206 INFO:teuthology.orchestra.run.smithi016.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-12T01:31:04.223 INFO:teuthology.orchestra.run.smithi016.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-12T01:31:04.230 INFO:teuthology.orchestra.run.smithi033.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-12T01:31:04.247 INFO:teuthology.orchestra.run.smithi033.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 733 kB/s | 31 kB 00:00 2024-09-12T01:31:04.264 INFO:teuthology.orchestra.run.smithi033.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-12T01:31:04.281 INFO:teuthology.orchestra.run.smithi033.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 2.1 MB/s | 36 kB 00:00 2024-09-12T01:31:04.306 INFO:teuthology.orchestra.run.smithi016.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-09-12T01:31:04.307 INFO:teuthology.orchestra.run.smithi033.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T01:31:04.332 INFO:teuthology.orchestra.run.smithi016.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-12T01:31:04.348 INFO:teuthology.orchestra.run.smithi033.stdout:(141/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.4 MB/s | 2.0 MB 00:00 2024-09-12T01:31:04.357 INFO:teuthology.orchestra.run.smithi016.stdout:(89/156): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-09-12T01:31:04.374 INFO:teuthology.orchestra.run.smithi033.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 807 kB/s | 74 kB 00:00 2024-09-12T01:31:04.382 INFO:teuthology.orchestra.run.smithi016.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-12T01:31:04.399 INFO:teuthology.orchestra.run.smithi033.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T01:31:04.407 INFO:teuthology.orchestra.run.smithi016.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-09-12T01:31:04.433 INFO:teuthology.orchestra.run.smithi016.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 709 kB/s | 54 kB 00:00 2024-09-12T01:31:04.441 INFO:teuthology.orchestra.run.smithi033.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-12T01:31:04.466 INFO:teuthology.orchestra.run.smithi016.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-09-12T01:31:04.467 INFO:teuthology.orchestra.run.smithi033.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 297 kB/s | 20 kB 00:00 2024-09-12T01:31:04.492 INFO:teuthology.orchestra.run.smithi016.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 755 kB/s | 44 kB 00:00 2024-09-12T01:31:04.492 INFO:teuthology.orchestra.run.smithi033.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T01:31:04.517 INFO:teuthology.orchestra.run.smithi033.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-12T01:31:04.525 INFO:teuthology.orchestra.run.smithi016.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T01:31:04.550 INFO:teuthology.orchestra.run.smithi016.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-09-12T01:31:04.567 INFO:teuthology.orchestra.run.smithi033.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T01:31:04.601 INFO:teuthology.orchestra.run.smithi016.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-09-12T01:31:04.626 INFO:teuthology.orchestra.run.smithi016.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.9 MB/s | 304 kB 00:00 2024-09-12T01:31:04.668 INFO:teuthology.orchestra.run.smithi016.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 862 kB/s | 58 kB 00:00 2024-09-12T01:31:04.693 INFO:teuthology.orchestra.run.smithi016.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-12T01:31:04.727 INFO:teuthology.orchestra.run.smithi016.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-12T01:31:04.751 INFO:teuthology.orchestra.run.smithi033.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-12T01:31:04.752 INFO:teuthology.orchestra.run.smithi016.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-09-12T01:31:04.789 INFO:teuthology.orchestra.run.smithi016.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 380 kB/s | 22 kB 00:00 2024-09-12T01:31:04.845 INFO:teuthology.orchestra.run.smithi016.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-12T01:31:04.876 INFO:teuthology.orchestra.run.smithi033.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.8 MB/s | 3.2 MB 00:00 2024-09-12T01:31:04.878 INFO:teuthology.orchestra.run.smithi016.stdout:(105/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-09-12T01:31:04.912 INFO:teuthology.orchestra.run.smithi016.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-09-12T01:31:04.937 INFO:teuthology.orchestra.run.smithi016.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 331 kB/s | 19 kB 00:00 2024-09-12T01:31:04.943 INFO:teuthology.orchestra.run.smithi033.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 613 kB/s | 41 kB 00:00 2024-09-12T01:31:04.963 INFO:teuthology.orchestra.run.smithi016.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T01:31:04.988 INFO:teuthology.orchestra.run.smithi016.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T01:31:05.013 INFO:teuthology.orchestra.run.smithi016.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T01:31:05.027 INFO:teuthology.orchestra.run.smithi033.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.9 MB/s | 249 kB 00:00 2024-09-12T01:31:05.038 INFO:teuthology.orchestra.run.smithi016.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-12T01:31:05.085 INFO:teuthology.orchestra.run.smithi033.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 4.6 MB/s | 2.3 MB 00:00 2024-09-12T01:31:05.128 INFO:teuthology.orchestra.run.smithi033.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 4.8 MB/s | 1.8 MB 00:00 2024-09-12T01:31:05.203 INFO:teuthology.orchestra.run.smithi033.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 20 MB/s | 2.4 MB 00:00 2024-09-12T01:31:05.205 INFO:teuthology.orchestra.run.smithi016.stdout:(112/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-12T01:31:05.253 INFO:teuthology.orchestra.run.smithi033.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 6.0 MB/s | 1.4 MB 00:00 2024-09-12T01:31:05.259 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:31:05.259 INFO:teuthology.orchestra.run.smithi033.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-12T01:31:05.264 INFO:teuthology.orchestra.run.smithi016.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-09-12T01:31:05.289 INFO:teuthology.orchestra.run.smithi016.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T01:31:05.314 INFO:teuthology.orchestra.run.smithi016.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T01:31:05.340 INFO:teuthology.orchestra.run.smithi016.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 559 kB/s | 14 kB 00:00 2024-09-12T01:31:05.365 INFO:teuthology.orchestra.run.smithi016.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-12T01:31:05.390 INFO:teuthology.orchestra.run.smithi016.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-09-12T01:31:05.424 INFO:teuthology.orchestra.run.smithi016.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-12T01:31:05.477 INFO:teuthology.orchestra.run.smithi016.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-12T01:31:05.483 INFO:teuthology.orchestra.run.smithi016.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-12T01:31:05.517 INFO:teuthology.orchestra.run.smithi016.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-12T01:31:05.534 INFO:teuthology.orchestra.run.smithi016.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 644 kB/s | 11 kB 00:00 2024-09-12T01:31:05.550 INFO:teuthology.orchestra.run.smithi016.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-12T01:31:05.567 INFO:teuthology.orchestra.run.smithi016.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-12T01:31:05.593 INFO:teuthology.orchestra.run.smithi016.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-12T01:31:05.618 INFO:teuthology.orchestra.run.smithi016.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 786 kB/s | 20 kB 00:00 2024-09-12T01:31:05.702 INFO:teuthology.orchestra.run.smithi016.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-09-12T01:31:05.727 INFO:teuthology.orchestra.run.smithi016.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T01:31:05.753 INFO:teuthology.orchestra.run.smithi016.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T01:31:05.777 INFO:teuthology.orchestra.run.smithi016.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T01:31:05.811 INFO:teuthology.orchestra.run.smithi016.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-09-12T01:31:05.837 INFO:teuthology.orchestra.run.smithi016.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-12T01:31:05.878 INFO:teuthology.orchestra.run.smithi016.stdout:(134/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-09-12T01:31:05.903 INFO:teuthology.orchestra.run.smithi016.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-12T01:31:05.929 INFO:teuthology.orchestra.run.smithi016.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-12T01:31:05.954 INFO:teuthology.orchestra.run.smithi016.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 767 kB/s | 90 kB 00:00 2024-09-12T01:31:05.979 INFO:teuthology.orchestra.run.smithi016.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-09-12T01:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-12T01:31:06.030 INFO:teuthology.orchestra.run.smithi016.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-12T01:31:06.055 INFO:teuthology.orchestra.run.smithi016.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-12T01:31:06.081 INFO:teuthology.orchestra.run.smithi016.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-12T01:31:06.106 INFO:teuthology.orchestra.run.smithi016.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 3.6 MB/s | 90 kB 00:00 2024-09-12T01:31:06.131 INFO:teuthology.orchestra.run.smithi016.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 806 kB/s | 20 kB 00:00 2024-09-12T01:31:06.157 INFO:teuthology.orchestra.run.smithi016.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-12T01:31:06.190 INFO:teuthology.orchestra.run.smithi016.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-12T01:31:06.224 INFO:teuthology.orchestra.run.smithi016.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-12T01:31:06.282 INFO:teuthology.orchestra.run.smithi016.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-12T01:31:06.479 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-09-12T01:31:06.483 INFO:teuthology.orchestra.run.smithi016.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 12 MB/s | 3.2 MB 00:00 2024-09-12T01:31:06.541 INFO:teuthology.orchestra.run.smithi016.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 9.8 MB/s | 3.4 MB 00:00 2024-09-12T01:31:06.633 INFO:teuthology.orchestra.run.smithi016.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 6.6 MB/s | 2.3 MB 00:00 2024-09-12T01:31:06.647 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-09-12T01:31:06.647 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-09-12T01:31:06.658 INFO:teuthology.orchestra.run.smithi016.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 349 kB/s | 41 kB 00:00 2024-09-12T01:31:06.701 INFO:teuthology.orchestra.run.smithi016.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 3.6 MB/s | 249 kB 00:00 2024-09-12T01:31:06.743 INFO:teuthology.orchestra.run.smithi016.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 7.0 MB/s | 1.8 MB 00:00 2024-09-12T01:31:06.884 INFO:teuthology.orchestra.run.smithi016.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-09-12T01:31:06.976 INFO:teuthology.orchestra.run.smithi016.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 4.3 MB/s | 1.4 MB 00:00 2024-09-12T01:31:06.981 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:31:06.982 INFO:teuthology.orchestra.run.smithi016.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-12T01:31:08.265 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-12T01:31:08.435 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-12T01:31:08.436 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-12T01:31:08.979 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-09-12T01:31:08.980 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-09-12T01:31:10.774 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-12T01:31:10.774 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-12T01:31:15.594 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-09-12T01:31:15.715 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T01:31:15.764 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T01:31:15.835 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T01:31:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T01:31:15.899 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T01:31:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T01:31:16.308 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T01:31:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:31:16.551 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:31:16.585 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:31:16.678 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:31:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:31:16.903 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T01:31:16.962 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T01:31:16.997 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:31:17.111 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:31:17.140 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:31:17.203 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:31:17.233 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:31:17.255 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:31:17.446 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:31:17.553 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-12T01:31:17.632 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:31:17.672 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T01:31:17.711 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T01:31:17.739 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T01:31:17.765 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T01:31:17.798 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T01:31:17.823 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T01:31:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:31:17.877 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T01:31:17.949 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:31:18.178 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:31:18.237 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T01:31:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T01:31:18.344 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:31:18.388 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T01:31:18.421 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T01:31:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T01:31:18.470 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:31:18.495 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:31:18.548 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T01:31:18.581 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:31:18.608 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T01:31:18.609 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:31:18.647 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T01:31:18.691 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T01:31:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T01:31:18.759 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T01:31:18.810 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T01:31:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T01:31:18.864 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T01:31:18.883 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T01:31:18.900 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:31:18.913 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T01:31:18.956 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T01:31:19.006 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:31:19.043 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:31:19.043 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T01:31:19.075 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T01:31:19.109 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:31:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T01:31:19.144 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:31:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T01:31:19.170 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:31:19.254 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T01:31:19.295 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T01:31:19.313 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:31:19.374 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T01:31:19.500 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:31:19.519 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T01:31:19.590 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T01:31:19.634 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T01:31:19.735 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:31:19.835 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:31:20.075 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:31:20.163 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T01:31:20.195 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T01:31:20.246 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T01:31:20.289 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T01:31:20.292 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T01:31:20.324 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T01:31:20.340 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T01:31:20.359 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T01:31:20.387 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T01:31:20.429 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T01:31:20.435 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T01:31:20.495 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T01:31:20.534 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T01:31:20.578 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T01:31:20.593 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T01:31:20.610 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T01:31:20.638 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T01:31:20.664 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T01:31:20.711 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T01:31:20.721 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T01:31:20.763 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T01:31:20.792 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T01:31:20.796 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T01:31:20.835 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T01:31:20.840 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T01:31:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T01:31:20.922 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T01:31:20.933 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T01:31:20.954 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T01:31:20.968 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T01:31:20.998 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T01:31:21.011 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T01:31:21.026 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T01:31:21.053 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T01:31:21.101 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T01:31:21.150 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T01:31:21.196 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T01:31:21.228 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T01:31:21.375 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T01:31:21.446 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T01:31:21.447 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T01:31:21.616 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T01:31:21.733 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T01:31:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T01:31:21.812 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T01:31:21.848 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T01:31:21.873 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T01:31:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T01:31:22.009 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T01:31:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T01:31:22.118 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T01:31:22.138 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T01:31:22.160 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T01:31:22.175 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T01:31:22.204 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T01:31:22.217 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T01:31:22.242 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T01:31:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T01:31:22.285 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T01:31:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T01:31:22.335 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T01:31:22.378 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T01:31:22.411 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T01:31:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T01:31:22.460 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T01:31:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T01:31:22.520 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T01:31:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T01:31:22.585 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T01:31:22.627 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T01:31:22.657 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T01:31:22.667 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T01:31:22.704 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T01:31:22.741 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T01:31:22.765 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T01:31:22.788 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T01:31:22.823 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T01:31:22.825 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T01:31:22.867 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T01:31:22.918 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T01:31:23.071 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T01:31:23.312 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T01:31:23.380 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T01:31:23.490 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T01:31:23.528 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T01:31:23.569 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T01:31:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T01:31:23.601 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T01:31:23.626 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T01:31:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T01:31:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T01:31:23.723 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T01:31:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T01:31:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T01:31:23.858 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T01:31:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T01:31:23.900 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T01:31:23.929 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T01:31:23.966 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T01:31:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T01:31:24.002 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T01:31:24.016 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T01:31:24.042 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T01:31:24.080 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T01:31:24.135 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T01:31:24.168 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T01:31:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T01:31:24.210 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T01:31:24.244 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T01:31:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T01:31:24.335 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T01:31:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T01:31:24.392 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T01:31:24.422 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T01:31:24.539 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T01:31:24.592 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T01:31:25.020 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T01:31:25.141 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T01:31:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T01:31:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T01:31:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T01:31:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T01:31:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T01:31:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T01:31:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T01:31:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T01:31:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T01:31:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T01:31:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T01:31:26.464 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T01:31:26.544 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T01:31:26.572 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T01:31:26.606 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T01:31:26.935 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T01:31:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:31:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:31:27.123 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T01:31:27.129 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T01:31:27.163 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T01:31:27.269 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T01:31:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T01:31:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:31:27.767 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:31:27.798 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T01:31:28.612 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T01:31:28.688 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T01:31:28.717 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T01:31:28.752 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T01:31:29.077 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T01:31:29.121 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:31:29.205 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:31:29.235 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T01:31:29.271 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T01:31:29.777 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T01:31:29.824 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:31:29.895 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:31:29.925 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T01:31:30.203 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:31:30.236 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:31:30.828 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:31:30.873 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:31:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:31:30.964 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T01:31:30.964 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T01:31:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:31:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:31:32.385 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:31:32.422 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:31:33.003 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:31:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:31:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:31:33.117 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T01:31:33.117 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T01:31:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:31:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:31:43.155 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:31:43.155 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-09-12T01:31:43.156 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-09-12T01:31:43.156 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-09-12T01:31:43.156 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-09-12T01:31:43.156 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-09-12T01:31:43.156 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /root 2024-09-12T01:31:43.156 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-09-12T01:31:43.156 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:31:43.351 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:31:43.368 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:31:43.368 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:43.368 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T01:31:43.368 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:31:44.904 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:31:44.932 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:31:44.933 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:44.933 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T01:31:44.933 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:31:44.933 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:31:44.933 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:31:45.171 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:31:45.196 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:31:45.196 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:45.196 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T01:31:45.196 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:31:45.196 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:31:45.196 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:31:45.655 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:31:45.680 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:31:45.680 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:45.680 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T01:31:45.680 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:31:45.680 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:31:45.680 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:31:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:31:45.766 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-09-12T01:31:45.766 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-09-12T01:31:45.767 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-09-12T01:31:45.767 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-09-12T01:31:45.767 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-09-12T01:31:45.767 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-09-12T01:31:45.767 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-09-12T01:31:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:31:45.786 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T01:31:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T01:31:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T01:31:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:31:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:31:45.970 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:45.970 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T01:31:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:31:46.101 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T01:31:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T01:31:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T01:31:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T01:31:46.477 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T01:31:46.524 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T01:31:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T01:31:46.582 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T01:31:47.052 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:31:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:31:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T01:31:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T01:31:47.179 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:31:47.505 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:31:47.533 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:31:47.533 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:47.534 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T01:31:47.534 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:31:47.534 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:31:47.534 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:31:47.763 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:31:47.788 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:31:47.789 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:47.789 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T01:31:47.789 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:31:47.789 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:31:47.789 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:31:48.256 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:31:48.281 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:31:48.281 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:48.281 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T01:31:48.281 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:31:48.282 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:31:48.282 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:31:48.282 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:31:48.314 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:31:48.387 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T01:31:48.446 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T01:31:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T01:31:48.706 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T01:31:48.767 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T01:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T01:31:48.945 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T01:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T01:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T01:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T01:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T01:31:49.330 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:31:49.364 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:31:49.473 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:31:49.642 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T01:31:49.667 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:31:49.691 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:31:49.691 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:49.691 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T01:31:49.692 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:31:49.692 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:31:49.692 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:31:49.732 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T01:31:49.743 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:31:49.754 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T01:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:31:49.847 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T01:31:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T01:31:50.884 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T01:31:50.919 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:31:50.937 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:31:50.944 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:31:50.944 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:50.945 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T01:31:50.945 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:31:50.945 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:31:50.945 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:31:50.975 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:31:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:31:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:31:51.011 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:51.011 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T01:31:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:31:51.070 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:31:51.097 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:31:51.097 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:51.097 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T01:31:51.097 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:31:51.097 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:31:51.097 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:31:51.438 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:31:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:31:51.462 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:51.463 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T01:31:51.463 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:31:51.463 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:31:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:31:51.994 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:31:52.025 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:31:52.134 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:31:52.303 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T01:31:52.335 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:31:52.361 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:31:52.361 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:52.361 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T01:31:52.361 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:31:52.361 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:31:52.361 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:31:52.420 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:31:52.432 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:31:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T01:31:52.588 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T01:31:53.563 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T01:31:53.589 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:31:53.614 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:31:53.614 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:53.614 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T01:31:53.614 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:31:53.615 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:31:53.615 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:31:53.656 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:31:53.681 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:31:53.681 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:53.681 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T01:31:53.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:31:53.732 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:31:53.757 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:31:53.757 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:53.757 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T01:31:53.757 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:31:53.757 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:31:53.757 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:31:54.074 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:31:54.099 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:31:54.100 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:31:54.100 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T01:31:54.100 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:31:54.100 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:31:54.101 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:31:56.301 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T01:31:56.343 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T01:31:56.384 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T01:31:56.415 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T01:31:56.442 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T01:31:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T01:31:56.697 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T01:31:56.727 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T01:31:56.757 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T01:31:56.802 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T01:31:57.440 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:31:57.584 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:31:57.605 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:31:58.512 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T01:31:58.597 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T01:31:58.644 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T01:31:58.736 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T01:31:58.774 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T01:31:58.800 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T01:31:58.826 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T01:31:58.826 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:31:58.910 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:31:58.910 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:31:58.962 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T01:31:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:31:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T01:31:59.021 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T01:31:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:31:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:31:59.062 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T01:31:59.093 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T01:31:59.120 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T01:31:59.221 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:31:59.221 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:31:59.305 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:31:59.334 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T01:31:59.374 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:31:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T01:31:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T01:31:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T01:31:59.481 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T01:32:00.112 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:32:00.155 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:32:00.177 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:32:01.104 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T01:32:01.202 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T01:32:01.255 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T01:32:01.349 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T01:32:01.397 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T01:32:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T01:32:01.456 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T01:32:01.457 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:32:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:32:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:32:01.633 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:32:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T01:32:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:32:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:32:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:32:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:32:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:32:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:32:05.119 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:32:05.119 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T01:32:05.119 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T01:32:05.119 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T01:32:05.119 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T01:32:05.119 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T01:32:05.119 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T01:32:05.119 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T01:32:05.119 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:32:05.119 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:32:05.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:32:05.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T01:32:05.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T01:32:05.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:32:05.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T01:32:05.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T01:32:05.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T01:32:05.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T01:32:05.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T01:32:05.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T01:32:05.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T01:32:05.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T01:32:05.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T01:32:05.121 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T01:32:05.121 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T01:32:05.121 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T01:32:05.121 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T01:32:05.121 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T01:32:05.121 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T01:32:05.121 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T01:32:05.135 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T01:32:05.135 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T01:32:05.135 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T01:32:05.135 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T01:32:05.135 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T01:32:05.136 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T01:32:05.136 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T01:32:05.136 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T01:32:05.136 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T01:32:05.136 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T01:32:05.136 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T01:32:05.136 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T01:32:05.136 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T01:32:05.136 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T01:32:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T01:32:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T01:32:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T01:32:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T01:32:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T01:32:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T01:32:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T01:32:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T01:32:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T01:32:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T01:32:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T01:32:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T01:32:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T01:32:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T01:32:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T01:32:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T01:32:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T01:32:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T01:32:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T01:32:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T01:32:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T01:32:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T01:32:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T01:32:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T01:32:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T01:32:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T01:32:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T01:32:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T01:32:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T01:32:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T01:32:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T01:32:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T01:32:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T01:32:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T01:32:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T01:32:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T01:32:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T01:32:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T01:32:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T01:32:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T01:32:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T01:32:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T01:32:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T01:32:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T01:32:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T01:32:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T01:32:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T01:32:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T01:32:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T01:32:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T01:32:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T01:32:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T01:32:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T01:32:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T01:32:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T01:32:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T01:32:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T01:32:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T01:32:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T01:32:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T01:32:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T01:32:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T01:32:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T01:32:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T01:32:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T01:32:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T01:32:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T01:32:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T01:32:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T01:32:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T01:32:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T01:32:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T01:32:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T01:32:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T01:32:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T01:32:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T01:32:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T01:32:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T01:32:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T01:32:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T01:32:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T01:32:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T01:32:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T01:32:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T01:32:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T01:32:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T01:32:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T01:32:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T01:32:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T01:32:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T01:32:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T01:32:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T01:32:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T01:32:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T01:32:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T01:32:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T01:32:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T01:32:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T01:32:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T01:32:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T01:32:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T01:32:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T01:32:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T01:32:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T01:32:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T01:32:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T01:32:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T01:32:07.718 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:32:07.719 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:07.719 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-09-12T01:32:07.719 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T01:32:07.719 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T01:32:07.719 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T01:32:07.719 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T01:32:07.719 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.719 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.719 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T01:32:07.719 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T01:32:07.719 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-09-12T01:32:07.719 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T01:32:07.720 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T01:32:07.720 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T01:32:07.720 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T01:32:07.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:07.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:07.721 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:07.721 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:07.721 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:07.721 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:07.721 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.721 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.721 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:07.721 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.721 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.721 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.721 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:07.721 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:07.722 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.722 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.722 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:07.722 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T01:32:07.722 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T01:32:07.722 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T01:32:07.722 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T01:32:07.722 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T01:32:07.722 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T01:32:07.722 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T01:32:07.722 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T01:32:07.722 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T01:32:07.722 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T01:32:07.723 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T01:32:07.723 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.723 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.723 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.723 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T01:32:07.723 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T01:32:07.723 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T01:32:07.723 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T01:32:07.723 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T01:32:07.723 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.723 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.723 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T01:32:07.724 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.724 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T01:32:07.724 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T01:32:07.724 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T01:32:07.724 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T01:32:07.724 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T01:32:07.724 INFO:teuthology.orchestra.run.smithi033.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T01:32:07.724 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T01:32:07.724 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T01:32:07.724 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T01:32:07.724 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T01:32:07.724 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T01:32:07.725 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T01:32:07.725 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T01:32:07.725 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T01:32:07.725 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T01:32:07.725 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T01:32:07.725 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T01:32:07.725 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T01:32:07.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T01:32:07.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T01:32:07.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T01:32:07.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T01:32:07.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T01:32:07.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.726 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.726 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.726 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T01:32:07.726 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T01:32:07.726 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T01:32:07.726 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T01:32:07.726 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T01:32:07.726 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T01:32:07.726 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T01:32:07.727 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T01:32:07.727 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T01:32:07.727 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T01:32:07.727 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T01:32:07.727 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T01:32:07.727 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T01:32:07.727 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T01:32:07.727 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T01:32:07.727 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T01:32:07.728 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T01:32:07.728 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T01:32:07.728 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T01:32:07.728 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T01:32:07.728 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T01:32:07.728 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T01:32:07.728 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T01:32:07.728 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T01:32:07.728 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T01:32:07.728 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T01:32:07.728 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T01:32:07.728 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T01:32:07.728 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T01:32:07.729 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T01:32:07.729 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T01:32:07.729 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T01:32:07.729 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T01:32:07.729 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T01:32:07.729 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T01:32:07.729 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T01:32:07.729 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T01:32:07.729 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T01:32:07.729 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T01:32:07.729 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T01:32:07.729 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.729 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T01:32:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T01:32:07.731 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T01:32:07.731 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T01:32:07.731 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T01:32:07.731 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T01:32:07.731 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T01:32:07.731 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.731 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.731 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:07.731 INFO:teuthology.orchestra.run.smithi033.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T01:32:07.731 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T01:32:07.731 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T01:32:07.732 INFO:teuthology.orchestra.run.smithi033.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T01:32:07.732 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T01:32:07.732 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T01:32:07.732 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T01:32:07.732 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:07.732 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-09-12T01:32:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:32:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T01:32:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T01:32:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T01:32:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T01:32:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T01:32:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T01:32:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T01:32:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T01:32:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T01:32:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T01:32:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T01:32:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T01:32:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T01:32:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T01:32:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T01:32:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T01:32:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T01:32:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T01:32:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T01:32:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T01:32:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T01:32:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T01:32:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T01:32:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T01:32:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T01:32:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T01:32:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T01:32:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T01:32:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T01:32:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T01:32:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T01:32:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T01:32:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T01:32:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T01:32:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T01:32:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T01:32:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T01:32:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T01:32:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T01:32:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T01:32:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T01:32:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T01:32:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T01:32:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T01:32:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T01:32:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T01:32:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T01:32:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T01:32:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T01:32:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T01:32:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T01:32:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T01:32:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T01:32:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T01:32:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T01:32:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T01:32:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T01:32:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T01:32:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T01:32:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T01:32:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T01:32:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T01:32:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T01:32:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T01:32:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T01:32:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T01:32:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T01:32:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T01:32:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T01:32:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T01:32:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T01:32:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T01:32:08.229 DEBUG:teuthology.parallel:result is None 2024-09-12T01:32:10.533 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:32:10.533 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:10.533 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-09-12T01:32:10.533 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T01:32:10.533 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T01:32:10.533 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T01:32:10.533 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T01:32:10.533 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.533 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.533 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T01:32:10.533 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.534 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:10.535 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:10.535 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:10.535 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:10.535 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:10.535 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.535 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.535 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:10.535 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.535 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.535 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.535 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:10.535 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:32:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T01:32:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T01:32:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T01:32:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T01:32:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T01:32:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T01:32:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T01:32:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T01:32:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T01:32:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T01:32:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T01:32:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T01:32:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T01:32:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T01:32:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T01:32:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T01:32:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T01:32:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T01:32:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T01:32:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T01:32:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T01:32:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T01:32:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T01:32:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T01:32:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T01:32:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T01:32:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T01:32:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T01:32:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T01:32:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T01:32:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T01:32:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T01:32:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T01:32:10.541 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T01:32:10.541 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T01:32:10.541 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T01:32:10.541 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T01:32:10.541 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T01:32:10.541 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T01:32:10.541 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T01:32:10.541 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T01:32:10.541 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T01:32:10.541 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T01:32:10.541 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T01:32:10.541 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T01:32:10.541 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T01:32:10.542 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T01:32:10.542 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T01:32:10.542 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T01:32:10.542 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T01:32:10.542 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T01:32:10.542 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T01:32:10.542 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T01:32:10.542 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T01:32:10.542 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T01:32:10.544 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T01:32:10.544 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T01:32:10.544 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T01:32:10.544 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.544 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.544 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T01:32:10.544 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T01:32:10.544 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T01:32:10.544 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.544 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T01:32:10.544 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T01:32:10.544 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T01:32:10.544 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.545 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:32:10.546 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T01:32:10.546 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T01:32:10.546 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T01:32:10.546 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T01:32:10.546 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T01:32:10.546 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T01:32:10.546 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T01:32:10.547 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:10.547 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-12T01:32:10.950 DEBUG:teuthology.parallel:result is None 2024-09-12T01:32:10.951 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:32:11.109 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T01:32:11.132 INFO:teuthology.orchestra.run.smithi016.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T01:32:11.133 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T01:32:11.133 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T01:32:11.135 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:32:11.288 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T01:32:11.316 INFO:teuthology.orchestra.run.smithi033.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T01:32:11.316 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T01:32:11.316 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T01:32:11.319 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T01:32:11.320 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:11.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T01:32:11.350 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:11.350 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T01:32:11.389 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T01:32:11.390 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:11.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T01:32:11.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T01:32:11.484 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:11.484 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T01:32:11.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T01:32:11.583 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T01:32:11.584 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:11.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T01:32:11.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T01:32:11.676 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:11.676 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T01:32:11.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T01:32:11.773 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T01:32:11.773 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:11.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T01:32:11.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T01:32:11.867 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:11.867 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T01:32:11.894 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T01:32:11.961 INFO:teuthology.run_tasks:Running task ceph... 2024-09-12T01:32:12.045 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-12T01:32:12.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2024-09-12T01:32:12.048 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /var/log/ceph 2024-09-12T01:32:12.075 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-12T01:32:12.075 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-12T01:32:12.114 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-12T01:32:12.142 INFO:tasks.ceph:Creating extra log directories... 2024-09-12T01:32:12.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-12T01:32:12.180 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-12T01:32:12.214 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-12T01:32:12.214 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-12T01:32:12.215 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-12T01:32:12.215 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:12.215 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-12T01:32:12.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-12T01:32:12.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-12T01:32:12.402 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:12.403 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-12T01:32:12.420 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-12T01:32:12.541 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-12T01:32:12.575 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-12T01:32:12.575 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Replacing daemon mds.a'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-12T01:32:12.575 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901553', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/cap-flush}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7901553', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Replacing daemon mds.a'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 481, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'sleep_before_teardown': 0, 'subset': '57/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLt9CBzOmnphNpGK/v5LTU3I4Fzbo5tmPyG527ZzDaKzn60hy2IxSBJ2Ygs4di5r04U8vrfhFtTpInS1b6OrXow=', 'smithi033.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCm9ElKSnb31Xz2HtSRvkQ/nVfeOWm02tjsH5mhjNuZNVvOO3JQvxMXQQDxByqajQd0iocTiwgBvENALH1R4KXg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Replacing daemon mds.a'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cap_flush']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-11_21:24:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-12T01:32:12.575 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-12T01:32:12.578 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-12T01:32:12.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T01:32:12.634 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T01:32:12.694 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:12.694 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T01:32:12.715 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T01:32:12.715 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-09-12T01:32:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T01:32:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:32:12.771 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-12T01:32:12.771 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:32:12.771 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:32:12.772 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 01:32:02.377939017 +0000 2024-09-12T01:32:12.772 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 01:30:13.682014749 +0000 2024-09-12T01:32:12.772 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 01:30:13.682014749 +0000 2024-09-12T01:32:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 01:30:13.682014749 +0000 2024-09-12T01:32:12.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T01:32:12.835 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-12T01:32:12.835 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-12T01:32:12.836 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 8.8431e-05 s, 5.8 MB/s 2024-09-12T01:32:12.836 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T01:32:12.893 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-09-12T01:32:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T01:32:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:32:12.949 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T01:32:12.949 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:32:12.949 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:32:12.949 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 01:32:02.377939017 +0000 2024-09-12T01:32:12.949 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 01:30:14.010009360 +0000 2024-09-12T01:32:12.949 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 01:30:14.010009360 +0000 2024-09-12T01:32:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 01:30:14.010009360 +0000 2024-09-12T01:32:12.950 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T01:32:13.013 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-12T01:32:13.013 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-12T01:32:13.013 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 8.5129e-05 s, 6.0 MB/s 2024-09-12T01:32:13.014 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T01:32:13.071 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-09-12T01:32:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T01:32:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:32:13.127 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-12T01:32:13.127 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:32:13.127 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:32:13.128 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 01:32:02.378939000 +0000 2024-09-12T01:32:13.128 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 01:30:14.338003971 +0000 2024-09-12T01:32:13.128 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 01:30:14.338003971 +0000 2024-09-12T01:32:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 01:30:14.338003971 +0000 2024-09-12T01:32:13.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T01:32:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-12T01:32:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-12T01:32:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.5458e-05 s, 5.4 MB/s 2024-09-12T01:32:13.193 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T01:32:13.250 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-09-12T01:32:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T01:32:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:32:13.306 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-12T01:32:13.307 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:32:13.307 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:32:13.307 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 01:32:02.378939000 +0000 2024-09-12T01:32:13.307 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 01:30:14.679998352 +0000 2024-09-12T01:32:13.307 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 01:30:14.679998352 +0000 2024-09-12T01:32:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 01:30:14.679998352 +0000 2024-09-12T01:32:13.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T01:32:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-12T01:32:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-12T01:32:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000100299 s, 5.1 MB/s 2024-09-12T01:32:13.371 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T01:32:13.428 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T01:32:13.428 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:13.428 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T01:32:13.448 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T01:32:13.449 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-09-12T01:32:13.507 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T01:32:13.508 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:32:13.508 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-12T01:32:13.508 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:32:13.508 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:32:13.508 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-12 01:31:59.761883281 +0000 2024-09-12T01:32:13.508 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-12 01:30:15.165642382 +0000 2024-09-12T01:32:13.508 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-12 01:30:15.165642382 +0000 2024-09-12T01:32:13.508 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-09-12 01:30:15.165642382 +0000 2024-09-12T01:32:13.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T01:32:13.577 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-12T01:32:13.577 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-12T01:32:13.577 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000272817 s, 1.9 MB/s 2024-09-12T01:32:13.578 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T01:32:13.639 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-09-12T01:32:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T01:32:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:32:13.698 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T01:32:13.698 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:32:13.698 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:32:13.698 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-12 01:31:59.761883281 +0000 2024-09-12T01:32:13.698 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-12 01:30:15.498637195 +0000 2024-09-12T01:32:13.698 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-12 01:30:15.498637195 +0000 2024-09-12T01:32:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-09-12 01:30:15.498637195 +0000 2024-09-12T01:32:13.698 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T01:32:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-12T01:32:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-12T01:32:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000211767 s, 2.4 MB/s 2024-09-12T01:32:13.767 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T01:32:13.826 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-09-12T01:32:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T01:32:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:32:13.885 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-12T01:32:13.885 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:32:13.885 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:32:13.885 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-12 01:31:59.761883281 +0000 2024-09-12T01:32:13.885 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-12 01:30:15.838631903 +0000 2024-09-12T01:32:13.885 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-12 01:30:15.838631903 +0000 2024-09-12T01:32:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-09-12 01:30:15.838631903 +0000 2024-09-12T01:32:13.886 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T01:32:13.951 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-12T01:32:13.952 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-12T01:32:13.952 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000219716 s, 2.3 MB/s 2024-09-12T01:32:13.953 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T01:32:14.011 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-09-12T01:32:14.068 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T01:32:14.068 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:32:14.068 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-12T01:32:14.068 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:32:14.068 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:32:14.068 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-12 01:31:59.762883266 +0000 2024-09-12T01:32:14.068 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-12 01:30:16.180626579 +0000 2024-09-12T01:32:14.068 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-12 01:30:16.180626579 +0000 2024-09-12T01:32:14.068 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-09-12 01:30:16.180626579 +0000 2024-09-12T01:32:14.069 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T01:32:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-12T01:32:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-12T01:32:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000291087 s, 1.8 MB/s 2024-09-12T01:32:14.135 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T01:32:14.193 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T01:32:14.193 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-12T01:32:14.193 INFO:tasks.ceph:Generating config... 2024-09-12T01:32:14.194 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-12T01:32:14.194 INFO:tasks.ceph:[client] debug client = 20 2024-09-12T01:32:14.194 INFO:tasks.ceph:[client] debug ms = 1 2024-09-12T01:32:14.194 INFO:tasks.ceph:[client] pid file = 2024-09-12T01:32:14.194 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-12T01:32:14.194 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-12T01:32:14.194 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-12T01:32:14.195 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-12T01:32:14.196 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-12T01:32:14.196 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-12T01:32:14.196 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-12T01:32:14.196 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-12T01:32:14.196 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-12T01:32:14.196 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-12T01:32:14.196 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-12T01:32:14.196 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-12T01:32:14.196 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-12T01:32:14.196 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-12T01:32:14.196 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-12T01:32:14.196 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-12T01:32:14.197 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-12T01:32:14.197 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-12T01:32:14.197 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-12T01:32:14.197 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-12T01:32:14.197 INFO:tasks.ceph:Setting up mon.a... 2024-09-12T01:32:14.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-12T01:32:14.252 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2024-09-12T01:32:14.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-12T01:32:14.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T01:32:14.332 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16'), ('mon.b', '172.21.15.33'), ('mon.c', '[v2:172.21.15.33:3301,v1:172.21.15.33:6790]')] 2024-09-12T01:32:14.333 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16,172.21.15.33,[v2:172.21.15.33:3301,v1:172.21.15.33:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-12T01:32:14.334 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:14.334 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T01:32:14.388 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --add b 172.21.15.33 --addv c '[v2:172.21.15.33:3301,v1:172.21.15.33:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:14.477 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:14.478 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid cb5903e4-2f7f-485b-9d92-173a6afaa2f8 2024-09-12T01:32:14.478 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-09-12T01:32:14.478 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2024-09-12T01:32:14.478 INFO:teuthology.orchestra.run.smithi016.stdout:fsid cb5903e4-2f7f-485b-9d92-173a6afaa2f8 2024-09-12T01:32:14.478 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2024-09-12T01:32:14.479284+0000 2024-09-12T01:32:14.478 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-12T01:32:14.479284+0000 2024-09-12T01:32:14.478 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 17 (quincy) 2024-09-12T01:32:14.478 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2024-09-12T01:32:14.478 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T01:32:14.478 INFO:teuthology.orchestra.run.smithi016.stdout:1: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-09-12T01:32:14.478 INFO:teuthology.orchestra.run.smithi016.stdout:2: [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] mon.c 2024-09-12T01:32:14.479 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-12T01:32:14.480 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T01:32:14.495 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID cb5903e4-2f7f-485b-9d92-173a6afaa2f8... 2024-09-12T01:32:14.496 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T01:32:14.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T01:32:14.580 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-12T01:32:14.580 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-09-12T01:32:14.580 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T01:32:14.580 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-09-12T01:32:14.580 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.580 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-09-12T01:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-09-12T01:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-09-12T01:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-09-12T01:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-09-12T01:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-09-12T01:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T01:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-09-12T01:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T01:32:14.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.33,[v2:172.21.15.33:3301,v1:172.21.15.33:6790]" 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = cb5903e4-2f7f-485b-9d92-173a6afaa2f8 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.583 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-09-12T01:32:14.584 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-09-12T01:32:14.584 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.584 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-09-12T01:32:14.584 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-09-12T01:32:14.584 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-09-12T01:32:14.584 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.584 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-09-12T01:32:14.584 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-09-12T01:32:14.584 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.585 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-09-12T01:32:14.585 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-09-12T01:32:14.585 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-09-12T01:32:14.585 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-09-12T01:32:14.585 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.585 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-09-12T01:32:14.585 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-09-12T01:32:14.586 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.586 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-09-12T01:32:14.586 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.586 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-09-12T01:32:14.586 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.586 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-09-12T01:32:14.586 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.586 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-09-12T01:32:14.586 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-09-12T01:32:14.587 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.587 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-09-12T01:32:14.587 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-09-12T01:32:14.587 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-09-12T01:32:14.587 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-09-12T01:32:14.587 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2024-09-12T01:32:14.587 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2024-09-12T01:32:14.587 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-12T01:32:14.588 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-09-12T01:32:14.588 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2024-09-12T01:32:14.588 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T01:32:14.588 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-09-12T01:32:14.588 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-09-12T01:32:14.588 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-09-12T01:32:14.588 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-09-12T01:32:14.588 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-12T01:32:14.589 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.589 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-09-12T01:32:14.589 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-12T01:32:14.589 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-09-12T01:32:14.589 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-12T01:32:14.589 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-12T01:32:14.589 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-12T01:32:14.589 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-12T01:32:14.590 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-09-12T01:32:14.590 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-12T01:32:14.590 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.590 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-09-12T01:32:14.590 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-12T01:32:14.590 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-12T01:32:14.591 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-09-12T01:32:14.591 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-12T01:32:14.591 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-09-12T01:32:14.591 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-09-12T01:32:14.591 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-12T01:32:14.591 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-09-12T01:32:14.591 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-09-12T01:32:14.591 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-12T01:32:14.591 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.591 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T01:32:14.591 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T01:32:14.592 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T01:32:14.592 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.592 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T01:32:14.592 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T01:32:14.592 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T01:32:14.593 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.593 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-09-12T01:32:14.593 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-09-12T01:32:14.593 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.593 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-09-12T01:32:14.593 INFO:teuthology.orchestra.run.smithi016.stdout: mon down mkfs grace = 300 2024-09-12T01:32:14.593 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-09-12T01:32:14.593 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on pool no app = False 2024-09-12T01:32:14.593 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-09-12T01:32:14.593 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-09-12T01:32:14.593 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-09-12T01:32:14.593 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-09-12T01:32:14.594 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T01:32:14.594 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T01:32:14.594 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-09-12T01:32:14.594 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-12T01:32:14.594 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-12T01:32:14.594 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-09-12T01:32:14.594 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-12T01:32:14.594 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-12T01:32:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-09-12T01:32:14.595 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2024-09-12T01:32:14.596 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-12T01:32:14.596 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-12T01:32:14.596 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-12T01:32:14.596 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-09-12T01:32:14.597 INFO:teuthology.orchestra.run.smithi033.stdout: chdir = "" 2024-09-12T01:32:14.597 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T01:32:14.597 INFO:teuthology.orchestra.run.smithi033.stdout: auth supported = cephx 2024-09-12T01:32:14.597 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.597 INFO:teuthology.orchestra.run.smithi033.stdout: filestore xattr use omap = true 2024-09-12T01:32:14.597 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.597 INFO:teuthology.orchestra.run.smithi033.stdout: mon clock drift allowed = 1.000 2024-09-12T01:32:14.597 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.597 INFO:teuthology.orchestra.run.smithi033.stdout: osd crush chooseleaf type = 0 2024-09-12T01:32:14.597 INFO:teuthology.orchestra.run.smithi033.stdout: auth debug = true 2024-09-12T01:32:14.597 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.597 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on old message = true 2024-09-12T01:32:14.597 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on bug = true 2024-09-12T01:32:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T01:32:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: mon pg warn max object skew = 0 2024-09-12T01:32:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T01:32:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T01:32:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default size = 2 2024-09-12T01:32:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow primary affinity = true 2024-09-12T01:32:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow pg remap = true 2024-09-12T01:32:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on legacy crush tunables = false 2024-09-12T01:32:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on crush straw calc version zero = false 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on no sortbitwise = false 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on osd down out interval zero = false 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on too few osds = false 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: mon_allow_pool_size_one = true 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: osd default data pool replay window = 5 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: mon allow pool delete = true 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.599 INFO:teuthology.orchestra.run.smithi033.stdout: mon cluster log file level = debug 2024-09-12T01:32:14.600 INFO:teuthology.orchestra.run.smithi033.stdout: debug asserts on shutdown = true 2024-09-12T01:32:14.600 INFO:teuthology.orchestra.run.smithi033.stdout: mon health detail to clog = false 2024-09-12T01:32:14.600 INFO:teuthology.orchestra.run.smithi033.stdout: mon host = "172.21.15.16,172.21.15.33,[v2:172.21.15.33:3301,v1:172.21.15.33:6790]" 2024-09-12T01:32:14.600 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = cb5903e4-2f7f-485b-9d92-173a6afaa2f8 2024-09-12T01:32:14.600 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.600 INFO:teuthology.orchestra.run.smithi033.stdout:[osd] 2024-09-12T01:32:14.601 INFO:teuthology.orchestra.run.smithi033.stdout: osd journal size = 100 2024-09-12T01:32:14.601 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.601 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub load threshold = 5.0 2024-09-12T01:32:14.601 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub max interval = 600 2024-09-12T01:32:14.601 INFO:teuthology.orchestra.run.smithi033.stdout: osd mclock profile = high_recovery_ops 2024-09-12T01:32:14.601 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.601 INFO:teuthology.orchestra.run.smithi033.stdout: osd recover clone overlap = true 2024-09-12T01:32:14.601 INFO:teuthology.orchestra.run.smithi033.stdout: osd recovery max chunk = 1048576 2024-09-12T01:32:14.601 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.601 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug shutdown = true 2024-09-12T01:32:14.601 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug op order = true 2024-09-12T01:32:14.601 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug verify stray on activate = true 2024-09-12T01:32:14.602 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug trim objects = true 2024-09-12T01:32:14.602 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.602 INFO:teuthology.orchestra.run.smithi033.stdout: osd open classes on start = true 2024-09-12T01:32:14.602 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug pg log writeout = true 2024-09-12T01:32:14.602 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.602 INFO:teuthology.orchestra.run.smithi033.stdout: osd deep scrub update digest min age = 30 2024-09-12T01:32:14.602 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.602 INFO:teuthology.orchestra.run.smithi033.stdout: osd map max advance = 10 2024-09-12T01:32:14.602 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.602 INFO:teuthology.orchestra.run.smithi033.stdout: journal zero on create = true 2024-09-12T01:32:14.603 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.603 INFO:teuthology.orchestra.run.smithi033.stdout: filestore ondisk finisher threads = 3 2024-09-12T01:32:14.603 INFO:teuthology.orchestra.run.smithi033.stdout: filestore apply finisher threads = 3 2024-09-12T01:32:14.603 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.603 INFO:teuthology.orchestra.run.smithi033.stdout: bdev debug aio = true 2024-09-12T01:32:14.603 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug misdirected ops = true 2024-09-12T01:32:14.604 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore block size = 96636764160 2024-09-12T01:32:14.604 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore fsck on mount = True 2024-09-12T01:32:14.604 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluefs = 20 2024-09-12T01:32:14.604 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluestore = 20 2024-09-12T01:32:14.604 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-09-12T01:32:14.604 INFO:teuthology.orchestra.run.smithi033.stdout: debug osd = 20 2024-09-12T01:32:14.605 INFO:teuthology.orchestra.run.smithi033.stdout: debug rocksdb = 10 2024-09-12T01:32:14.605 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T01:32:14.605 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd full ratio = 0.9 2024-09-12T01:32:14.605 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd nearfull ratio = 0.8 2024-09-12T01:32:14.605 INFO:teuthology.orchestra.run.smithi033.stdout: osd failsafe full ratio = 0.95 2024-09-12T01:32:14.605 INFO:teuthology.orchestra.run.smithi033.stdout: osd objectstore = bluestore 2024-09-12T01:32:14.605 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2024-09-12T01:32:14.605 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.606 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr] 2024-09-12T01:32:14.606 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-09-12T01:32:14.606 INFO:teuthology.orchestra.run.smithi033.stdout: debug mgr = 20 2024-09-12T01:32:14.606 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2024-09-12T01:32:14.606 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2024-09-12T01:32:14.606 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2024-09-12T01:32:14.606 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2024-09-12T01:32:14.606 INFO:teuthology.orchestra.run.smithi033.stdout: mgr/telemetry/nag = false 2024-09-12T01:32:14.606 INFO:teuthology.orchestra.run.smithi033.stdout: debug client = 20 2024-09-12T01:32:14.606 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.606 INFO:teuthology.orchestra.run.smithi033.stdout:[mon] 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: debug paxos = 20 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: mon data avail warn = 5 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: mon mgr mkfs grace = 240 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd reporter subtree level = osd 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd prime pg temp = true 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T01:32:14.607 INFO:teuthology.orchestra.run.smithi033.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: # 1m isn't quite enough 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: mon_down_mkfs_grace = 2m 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_filestore_osds = false 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: mon down mkfs grace = 300 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: mon op complaint time = 120 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on pool no app = False 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-12T01:32:14.608 INFO:teuthology.orchestra.run.smithi033.stdout:[client] 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout: rgw cache enabled = true 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable ops log = true 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable usage log = true 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout: client mount timeout = 600 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout: debug client = 20 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = "" 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.a] 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.b] 2024-09-12T01:32:14.609 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.c] 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c] 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d] 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout:[mds] 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout: debug mds = 20 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout: debug mds balancer = 20 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug frag = True 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug scatterstat = True 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout: mds op complaint time = 180 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout: mds verify scatter = True 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2024-09-12T01:32:14.610 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2024-09-12T01:32:14.612 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-12T01:32:14.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-12T01:32:14.708 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-12T01:32:14.708 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:14.708 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-12T01:32:14.724 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:14.724 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-12T01:32:14.780 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:32:14.780 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:14.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-12T01:32:14.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T01:32:14.855 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:14.855 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:14.911 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:32:14.911 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:14.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-12T01:32:14.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T01:32:14.954 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:14.955 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:15.013 INFO:tasks.ceph:Setting up mon nodes... 2024-09-12T01:32:15.013 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-12T01:32:15.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-12T01:32:15.066 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-12T01:32:15.069 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-12T01:32:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-12T01:32:15.140 INFO:tasks.ceph:Setting up mds nodes... 2024-09-12T01:32:15.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-12T01:32:15.193 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-12T01:32:15.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-12T01:32:15.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-12T01:32:15.314 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-12T01:32:15.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-12T01:32:15.342 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-12T01:32:15.401 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-12T01:32:15.403 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-12T01:32:15.430 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-12T01:32:15.525 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-12T01:32:15.528 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-12T01:32:15.554 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-12T01:32:15.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T01:32:15.597 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-12T01:32:15.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-12T01:32:15.692 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-12T01:32:15.706 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-12T01:32:15.749 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-12T01:32:15.762 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-12T01:32:15.844 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-12T01:32:15.857 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-12T01:32:15.858 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-12T01:32:15.858 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-12T01:32:15.884 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T01:32:15.884 INFO:tasks.ceph:role: osd.0 2024-09-12T01:32:15.885 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:32:15.885 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-12T01:32:15.950 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T01:32:15.950 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T01:32:15.951 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T01:32:15.951 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T01:32:15.951 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T01:32:15.951 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-12T01:32:15.951 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T01:32:15.952 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T01:32:15.952 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T01:32:15.952 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T01:32:15.983 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-12T01:32:15.985 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-12T01:32:15.986 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-12T01:32:16.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-12T01:32:16.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-12T01:32:16.195 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T01:32:16.195 INFO:tasks.ceph:role: osd.1 2024-09-12T01:32:16.195 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:32:16.195 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-12T01:32:16.261 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T01:32:16.261 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T01:32:16.261 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T01:32:16.261 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T01:32:16.261 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T01:32:16.261 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-12T01:32:16.261 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T01:32:16.261 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T01:32:16.262 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T01:32:16.262 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T01:32:16.294 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-12T01:32:16.296 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-12T01:32:16.296 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-12T01:32:16.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-12T01:32:16.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-12T01:32:16.502 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T01:32:16.502 INFO:tasks.ceph:role: osd.2 2024-09-12T01:32:16.502 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:32:16.502 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-12T01:32:16.568 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T01:32:16.568 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T01:32:16.568 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T01:32:16.568 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T01:32:16.568 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T01:32:16.568 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-12T01:32:16.568 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T01:32:16.568 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T01:32:16.568 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T01:32:16.569 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T01:32:16.600 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-12T01:32:16.603 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-12T01:32:16.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-12T01:32:16.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-12T01:32:16.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-12T01:32:16.813 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T01:32:16.814 INFO:tasks.ceph:role: osd.3 2024-09-12T01:32:16.814 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:32:16.814 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-12T01:32:16.878 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T01:32:16.879 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T01:32:16.879 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T01:32:16.879 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T01:32:16.879 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T01:32:16.879 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-12T01:32:16.879 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T01:32:16.879 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T01:32:16.879 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T01:32:16.879 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T01:32:16.911 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-12T01:32:16.913 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-12T01:32:16.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-12T01:32:16.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-12T01:32:17.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:17.148+0000 7f3ee773e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-12T01:32:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:17.148+0000 7f3ee773e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-12T01:32:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:17.148+0000 7f3ee773e640 -1 bdev(0x56444b909800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-12T01:32:17.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:17.149+0000 7f3ee773e640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-12T01:32:20.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-12T01:32:20.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:20.183+0000 7fdab8bbb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-12T01:32:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:20.183+0000 7fdab8bbb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-12T01:32:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:20.183+0000 7fdab8bbb640 -1 bdev(0x565072ee7800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-12T01:32:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:20.183+0000 7fdab8bbb640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-12T01:32:23.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-12T01:32:23.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:23.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:23.224+0000 7f9858fb2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-12T01:32:23.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:23.224+0000 7f9858fb2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-12T01:32:23.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:23.224+0000 7f9858fb2640 -1 bdev(0x5618b664b800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-12T01:32:23.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:23.224+0000 7f9858fb2640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-12T01:32:26.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-12T01:32:26.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:26.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:26.256+0000 7fa0628e0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-12T01:32:26.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:26.256+0000 7fa0628e0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-12T01:32:26.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:26.256+0000 7fa0628e0640 -1 bdev(0x557cba7e5800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-12T01:32:26.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:26.256+0000 7fa0628e0640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-12T01:32:29.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-12T01:32:29.213 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-12T01:32:29.245 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T01:32:29.245 INFO:tasks.ceph:role: osd.4 2024-09-12T01:32:29.245 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:32:29.245 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-12T01:32:29.323 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T01:32:29.324 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T01:32:29.324 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T01:32:29.324 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T01:32:29.324 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T01:32:29.324 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-09-12T01:32:29.324 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T01:32:29.324 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T01:32:29.324 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T01:32:29.324 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T01:32:29.832 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-09-12T01:32:29.835 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-09-12T01:32:29.835 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-12T01:32:29.935 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-12T01:32:29.968 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-12T01:32:30.035 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T01:32:30.035 INFO:tasks.ceph:role: osd.5 2024-09-12T01:32:30.035 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:32:30.036 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-12T01:32:30.109 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T01:32:30.109 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T01:32:30.109 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T01:32:30.109 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T01:32:30.109 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T01:32:30.109 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-09-12T01:32:30.109 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T01:32:30.109 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T01:32:30.109 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T01:32:30.109 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T01:32:30.567 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-09-12T01:32:30.570 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-09-12T01:32:30.570 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-12T01:32:30.661 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-12T01:32:30.691 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-12T01:32:30.757 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T01:32:30.757 INFO:tasks.ceph:role: osd.6 2024-09-12T01:32:30.757 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:32:30.757 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-12T01:32:30.831 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T01:32:30.831 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T01:32:30.831 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T01:32:30.831 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T01:32:30.831 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T01:32:30.831 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-09-12T01:32:30.831 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T01:32:30.832 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T01:32:30.832 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T01:32:30.832 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T01:32:31.316 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-09-12T01:32:31.318 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-09-12T01:32:31.318 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-12T01:32:31.445 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-12T01:32:31.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-12T01:32:31.542 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T01:32:31.543 INFO:tasks.ceph:role: osd.7 2024-09-12T01:32:31.543 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:32:31.543 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-12T01:32:31.617 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T01:32:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T01:32:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T01:32:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T01:32:31.617 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T01:32:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-09-12T01:32:31.618 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T01:32:31.618 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T01:32:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T01:32:31.618 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T01:32:32.143 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-09-12T01:32:32.145 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-09-12T01:32:32.145 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-12T01:32:32.285 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-12T01:32:32.322 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:32.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:32.421+0000 7f93e9c0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-12T01:32:32.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:32.421+0000 7f93e9c0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-12T01:32:32.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:32.421+0000 7f93e9c0d640 -1 bdev(0x55893f8ff800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-12T01:32:32.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:32.421+0000 7f93e9c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-12T01:32:35.324 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-12T01:32:35.356 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:35.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:35.448+0000 7f3139ee4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-12T01:32:35.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:35.449+0000 7f3139ee4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-12T01:32:35.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:35.449+0000 7f3139ee4640 -1 bdev(0x55a6de92d800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-12T01:32:35.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:35.449+0000 7f3139ee4640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-12T01:32:38.611 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-12T01:32:38.642 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:38.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:38.735+0000 7f1c78dc5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-12T01:32:38.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:38.735+0000 7f1c78dc5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-12T01:32:38.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:38.735+0000 7f1c78dc5640 -1 bdev(0x563362abf800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-12T01:32:38.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:38.735+0000 7f1c78dc5640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-12T01:32:41.888 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-12T01:32:41.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:42.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:42.013+0000 7f48fe69c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-12T01:32:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:42.013+0000 7f48fe69c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-12T01:32:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:42.013+0000 7f48fe69c640 -1 bdev(0x5600061ef800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-12T01:32:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:42.013+0000 7f48fe69c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-12T01:32:42.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:32:42.580 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:32:45.171 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-12T01:32:45.201 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-12T01:32:45.202 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:45.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-12T01:32:45.233 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:45.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-12T01:32:45.302 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:45.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-12T01:32:45.371 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:45.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-12T01:32:45.439 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:45.439 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-12T01:32:45.506 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:45.506 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-12T01:32:45.573 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:45.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-12T01:32:45.640 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:45.640 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-12T01:32:45.671 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:45.671 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-12T01:32:45.744 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:45.745 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-12T01:32:45.812 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:45.812 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-12T01:32:45.882 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:45.883 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-12T01:32:45.952 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:45.952 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-12T01:32:46.022 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:46.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-12T01:32:46.091 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:46.091 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-12T01:32:46.110 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:46.110 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-12T01:32:46.168 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:46.168 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-12T01:32:46.187 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:46.187 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-12T01:32:46.245 INFO:tasks.ceph:Adding keys to all mons... 2024-09-12T01:32:46.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-12T01:32:46.248 DEBUG:teuthology.orchestra.run.smithi033:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-12T01:32:46.288 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.y] 2024-09-12T01:32:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCfROJmG2gUBBAAdOEewBDrj7a7agWTq5zuWg== 2024-09-12T01:32:46.289 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-12T01:32:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCfROJmrQSeCxAA/0QxZw6X+WKa/poBP1nG3A== 2024-09-12T01:32:46.289 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-12T01:32:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCfROJm6nrNEhAAJ4H4+3KC1mWOT/d/eUWtew== 2024-09-12T01:32:46.289 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2024-09-12T01:32:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQChROJmKyXsCBAAsqnBx3QHSaRg1xdqA3GAqA== 2024-09-12T01:32:46.289 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2024-09-12T01:32:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCkROJmWcL9ChAAMF9aonFTkDzGd8LhRvZXTQ== 2024-09-12T01:32:46.289 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2024-09-12T01:32:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCnROJmW0JwDRAAoxlXba7CaNVGw8dKQwatdQ== 2024-09-12T01:32:46.290 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2024-09-12T01:32:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCqROJmM7VWDxAANSE5pwGw3KJDpAOvvncT2w== 2024-09-12T01:32:46.290 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2024-09-12T01:32:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCfROJmFh9NCBAAfVJ6iVOOWFnk/XdcEXgxfg== 2024-09-12T01:32:46.290 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-12T01:32:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCfROJmuc/8FxAA/R1oc2RPaidINdOrcjez8g== 2024-09-12T01:32:46.290 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-12T01:32:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCfROJmXdhlHxAAgc1TWA2F7amP6+mYD7nPNw== 2024-09-12T01:32:46.290 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.4] 2024-09-12T01:32:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCwROJmMu8iGRAAB8dmNJfx9qY7hxMPCqPd0Q== 2024-09-12T01:32:46.290 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.5] 2024-09-12T01:32:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCzROJmALnMGhAAAgYW8kCOG5jjw/mCSsI1vQ== 2024-09-12T01:32:46.291 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.6] 2024-09-12T01:32:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC2ROJmD1ncKxAAbewBA0yylXh538+uit6uWA== 2024-09-12T01:32:46.291 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.7] 2024-09-12T01:32:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC6ROJm9gXSABAABcCBjzd83OSzpRz1+k7Ggg== 2024-09-12T01:32:46.291 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-12T01:32:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCfROJmhMy2IxAAGVo/xGjEHxUMiQfBrw0mrQ== 2024-09-12T01:32:46.291 INFO:teuthology.orchestra.run.smithi016.stdout:[client.2] 2024-09-12T01:32:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCfROJmxcxiKRAAAf2Zq/oLEIvWNh8dAKxOww== 2024-09-12T01:32:46.291 INFO:teuthology.orchestra.run.smithi016.stdout:[client.1] 2024-09-12T01:32:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCfROJmfwDBLBAABsu2b/R1NCaH5DT29W8JQg== 2024-09-12T01:32:46.291 INFO:teuthology.orchestra.run.smithi016.stdout:[client.3] 2024-09-12T01:32:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCfROJmsgRxMhAA2TuosZ1pMaOtZQPwcoZZeg== 2024-09-12T01:32:46.313 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.y] 2024-09-12T01:32:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCfROJmG2gUBBAAdOEewBDrj7a7agWTq5zuWg== 2024-09-12T01:32:46.313 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2024-09-12T01:32:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCfROJmrQSeCxAA/0QxZw6X+WKa/poBP1nG3A== 2024-09-12T01:32:46.313 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c] 2024-09-12T01:32:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCfROJm6nrNEhAAJ4H4+3KC1mWOT/d/eUWtew== 2024-09-12T01:32:46.313 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.0] 2024-09-12T01:32:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQChROJmKyXsCBAAsqnBx3QHSaRg1xdqA3GAqA== 2024-09-12T01:32:46.313 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.1] 2024-09-12T01:32:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCkROJmWcL9ChAAMF9aonFTkDzGd8LhRvZXTQ== 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.2] 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCnROJmW0JwDRAAoxlXba7CaNVGw8dKQwatdQ== 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.3] 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCqROJmM7VWDxAANSE5pwGw3KJDpAOvvncT2w== 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.x] 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCfROJmFh9NCBAAfVJ6iVOOWFnk/XdcEXgxfg== 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCfROJmuc/8FxAA/R1oc2RPaidINdOrcjez8g== 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d] 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCfROJmXdhlHxAAgc1TWA2F7amP6+mYD7nPNw== 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.4] 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCwROJmMu8iGRAAB8dmNJfx9qY7hxMPCqPd0Q== 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.5] 2024-09-12T01:32:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCzROJmALnMGhAAAgYW8kCOG5jjw/mCSsI1vQ== 2024-09-12T01:32:46.315 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.6] 2024-09-12T01:32:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQC2ROJmD1ncKxAAbewBA0yylXh538+uit6uWA== 2024-09-12T01:32:46.315 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.7] 2024-09-12T01:32:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQC6ROJm9gXSABAABcCBjzd83OSzpRz1+k7Ggg== 2024-09-12T01:32:46.315 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-09-12T01:32:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCfROJmhMy2IxAAGVo/xGjEHxUMiQfBrw0mrQ== 2024-09-12T01:32:46.315 INFO:teuthology.orchestra.run.smithi033.stdout:[client.2] 2024-09-12T01:32:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCfROJmxcxiKRAAAf2Zq/oLEIvWNh8dAKxOww== 2024-09-12T01:32:46.315 INFO:teuthology.orchestra.run.smithi033.stdout:[client.1] 2024-09-12T01:32:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCfROJmfwDBLBAABsu2b/R1NCaH5DT29W8JQg== 2024-09-12T01:32:46.315 INFO:teuthology.orchestra.run.smithi033.stdout:[client.3] 2024-09-12T01:32:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCfROJmsgRxMhAA2TuosZ1pMaOtZQPwcoZZeg== 2024-09-12T01:32:46.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T01:32:46.335 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T01:32:46.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T01:32:46.416 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T01:32:46.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T01:32:46.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T01:32:46.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:46.535 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:46.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:46.589 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:46.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:46.644 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:46.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:46.698 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:46.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T01:32:46.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T01:32:46.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T01:32:46.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T01:32:46.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T01:32:46.863 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T01:32:46.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:46.921 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:46.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:46.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:47.029 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:47.032 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:47.084 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:47.086 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T01:32:47.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T01:32:47.141 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T01:32:47.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T01:32:47.198 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T01:32:47.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T01:32:47.256 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T01:32:47.312 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T01:32:47.314 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T01:32:47.369 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-12T01:32:47.369 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-12T01:32:47.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T01:32:47.493 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-12T01:32:47.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-12T01:32:47.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T01:32:47.643 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-12T01:32:47.668 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-12T01:32:47.733 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T01:32:47.831 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-12T01:32:47.857 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:47.860 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-12T01:32:47.913 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-12T01:32:47.914 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-12T01:32:47.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-12T01:32:47.916 INFO:tasks.ceph.mon.a:Started 2024-09-12T01:32:47.916 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-12T01:32:47.916 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-12T01:32:47.955 INFO:tasks.ceph.mon.b:Started 2024-09-12T01:32:47.955 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-12T01:32:47.956 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-12T01:32:47.958 INFO:tasks.ceph.mon.c:Started 2024-09-12T01:32:47.958 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-12T01:32:47.958 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-12T01:32:47.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-12T01:32:47.961 INFO:tasks.ceph.mgr.y:Started 2024-09-12T01:32:47.961 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-12T01:32:47.961 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-12T01:32:47.963 INFO:tasks.ceph.mgr.x:Started 2024-09-12T01:32:47.963 INFO:tasks.ceph:Setting crush tunables to default 2024-09-12T01:32:47.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2024-09-12T01:32:48.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.068+0000 7f0160bb7640 1 Processor -- start 2024-09-12T01:32:48.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.068+0000 7f0160bb7640 1 -- start start 2024-09-12T01:32:48.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.068+0000 7f0160bb7640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0cd310 0x7f015c1653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:48.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.068+0000 7f0160bb7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f015c0cfba0 con 0x7f015c0cf550 2024-09-12T01:32:48.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.069+0000 7f0160bb7640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f015c0cfe80 con 0x7f015c0cc550 2024-09-12T01:32:48.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.069+0000 7f0160bb7640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f015c0b4360 con 0x7f015c0cd310 2024-09-12T01:32:48.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.069+0000 7f0151d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f015c0cf550 0x7f015c0ccc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46004/0 (socket says 172.21.15.16:46004) 2024-09-12T01:32:48.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.069+0000 7f0151d9b640 1 -- 172.21.15.16:0/3994992760 learned_addr learned my addr 172.21.15.16:0/3994992760 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:32:48.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.069+0000 7f015ad9d640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0cd310 0x7f015c1653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:48.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.069+0000 7f015ad9d640 1 -- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0cd310 msgr2=0x7f015c1653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-12T01:32:48.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.069+0000 7f015ad9d640 1 -- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0cd310 msgr2=0x7f015c1653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-12T01:32:48.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.069+0000 7f015ad9d640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0cd310 0x7f015c1653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:32:48.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.069+0000 7f015ad9d640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0cd310 0x7f015c1653c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:32:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.227+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2364325800 0 0) 0x7f015c0cfe80 con 0x7f015c0cc550 2024-09-12T01:32:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.227+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0148003440 con 0x7f015c0cc550 2024-09-12T01:32:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 <== mon.1 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1393028414 0 0) 0x7f0148003440 con 0x7f015c0cc550 2024-09-12T01:32:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0cd310 msgr2=0x7f015c1653c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:32:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0159d9b640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0cd310 0x7f015c1653c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 >> v1:172.21.15.16:6789/0 conn(0x7f015c0cf550 legacy=0x7f015c0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c0b45b0 con 0x7f015c0cc550 2024-09-12T01:32:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 <== mon.1 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f014c001670 con 0x7f015c0cc550 2024-09-12T01:32:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 >> v1:172.21.15.33:6789/0 conn(0x7f015c0cc550 legacy=0x7f015c073b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0159d9b640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0148003aa0 0x7f0148003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0159d9b640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0148008950 0x7f014800ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0159d9b640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0cd310 0x7f014800d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f015c0b4360 con 0x7f0148003aa0 2024-09-12T01:32:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f014800de60 con 0x7f0148008950 2024-09-12T01:32:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f014800e060 con 0x7f015c0cd310 2024-09-12T01:32:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f015a59c640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0148003aa0 0x7f0148003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f0151d9b640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0148008950 0x7f014800ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f015ad9d640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0cd310 0x7f014800d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f015a59c640 1 -- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0cd310 msgr2=0x7f014800d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f015a59c640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0cd310 0x7f014800d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f015a59c640 1 -- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0148008950 msgr2=0x7f014800ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f015a59c640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0148008950 0x7f014800ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.228+0000 7f015a59c640 1 -- 172.21.15.16:0/3994992760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0148003440 con 0x7f0148003aa0 2024-09-12T01:32:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.229+0000 7f015a59c640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0148003aa0 0x7f0148003ee0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f014c00c0e0 tx=0x7f0148003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5460d3076ffb33d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.229+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c036070 con 0x7f0148003aa0 2024-09-12T01:32:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.229+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f014c031620 con 0x7f0148003aa0 2024-09-12T01:32:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.229+0000 7f0159d9b640 1 -- 172.21.15.16:0/3994992760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c007c20 con 0x7f0148003aa0 2024-09-12T01:32:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.230+0000 7f0160bb7640 1 -- 172.21.15.16:0/3994992760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0148003aa0 msgr2=0x7f0148003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.230+0000 7f0160bb7640 1 --2- 172.21.15.16:0/3994992760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0148003aa0 0x7f0148003ee0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f014c00c0e0 tx=0x7f0148003730 comp rx=0 tx=0).stop 2024-09-12T01:32:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.230+0000 7f015ad9d640 1 -- 172.21.15.16:0/3994992760 reap_dead start 2024-09-12T01:32:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.230+0000 7f0160bb7640 1 -- 172.21.15.16:0/3994992760 shutdown_connections 2024-09-12T01:32:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.230+0000 7f0160bb7640 1 -- 172.21.15.16:0/3994992760 >> 172.21.15.16:0/3994992760 conn(0x7f015c06baf0 msgr2=0x7f015c06bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.230+0000 7f0160bb7640 1 -- 172.21.15.16:0/3994992760 shutdown_connections 2024-09-12T01:32:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.230+0000 7f0160bb7640 1 -- 172.21.15.16:0/3994992760 wait complete. 2024-09-12T01:32:48.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.231+0000 7f0160bb7640 1 Processor -- start 2024-09-12T01:32:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.231+0000 7f0160bb7640 1 -- start start 2024-09-12T01:32:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.232+0000 7f0160bb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0148003aa0 0x7f015c157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.232+0000 7f0160bb7640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0b0690 0x7f015c0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.232+0000 7f0160bb7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f015c0b0b10 0x7f015c0c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.232+0000 7f0160bb7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f015c0b4db0 con 0x7f0148003aa0 2024-09-12T01:32:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.232+0000 7f0160bb7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f015c0cedd0 con 0x7f015c0b0b10 2024-09-12T01:32:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.232+0000 7f0160bb7640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f015c15dca0 con 0x7f015c0b0690 2024-09-12T01:32:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.232+0000 7f015a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0148003aa0 0x7f015c157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.232+0000 7f015a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0148003aa0 0x7f015c157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55046/0 (socket says 172.21.15.16:55046) 2024-09-12T01:32:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.232+0000 7f015a59c640 1 -- 172.21.15.16:0/2652805044 learned_addr learned my addr 172.21.15.16:0/2652805044 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:32:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.232+0000 7f0151d9b640 1 --2- 172.21.15.16:0/2652805044 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0b0690 0x7f015c0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.232+0000 7f015ad9d640 1 --2- 172.21.15.16:0/2652805044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f015c0b0b10 0x7f015c0c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.233+0000 7f015a59c640 1 -- 172.21.15.16:0/2652805044 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0b0690 msgr2=0x7f015c0c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:48.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.233+0000 7f015a59c640 1 --2- 172.21.15.16:0/2652805044 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0b0690 0x7f015c0c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.233+0000 7f015a59c640 1 -- 172.21.15.16:0/2652805044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f015c0b0b10 msgr2=0x7f015c0c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:48.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.233+0000 7f015a59c640 1 --2- 172.21.15.16:0/2652805044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f015c0b0b10 0x7f015c0c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.233+0000 7f015a59c640 1 -- 172.21.15.16:0/2652805044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c15d400 con 0x7f0148003aa0 2024-09-12T01:32:48.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.233+0000 7f015a59c640 1 --2- 172.21.15.16:0/2652805044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0148003aa0 0x7f015c157ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f014c000c00 tx=0x7f015c0d0050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:48.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.233+0000 7f01537fe640 1 -- 172.21.15.16:0/2652805044 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c041020 con 0x7f0148003aa0 2024-09-12T01:32:48.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.233+0000 7f01537fe640 1 -- 172.21.15.16:0/2652805044 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f014c037040 con 0x7f0148003aa0 2024-09-12T01:32:48.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.233+0000 7f0160bb7640 1 -- 172.21.15.16:0/2652805044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015c15d6e0 con 0x7f0148003aa0 2024-09-12T01:32:48.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.233+0000 7f01537fe640 1 -- 172.21.15.16:0/2652805044 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c036070 con 0x7f0148003aa0 2024-09-12T01:32:48.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.233+0000 7f0160bb7640 1 -- 172.21.15.16:0/2652805044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015c15aed0 con 0x7f0148003aa0 2024-09-12T01:32:48.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.233+0000 7f01537fe640 1 -- 172.21.15.16:0/2652805044 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f014c044020 con 0x7f0148003aa0 2024-09-12T01:32:48.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.233+0000 7f01537fe640 1 -- 172.21.15.16:0/2652805044 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f014c03d480 con 0x7f0148003aa0 2024-09-12T01:32:48.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.234+0000 7f0160bb7640 1 -- 172.21.15.16:0/2652805044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015c073b60 con 0x7f0148003aa0 2024-09-12T01:32:48.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.237+0000 7f01537fe640 1 -- 172.21.15.16:0/2652805044 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f014c04b030 con 0x7f0148003aa0 2024-09-12T01:32:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.301+0000 7f0160bb7640 1 -- 172.21.15.16:0/2652805044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f015c06e660 con 0x7f0148003aa0 2024-09-12T01:32:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.303+0000 7f01537fe640 1 -- 172.21.15.16:0/2652805044 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f014c003b10 con 0x7f0148003aa0 2024-09-12T01:32:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-09-12T01:32:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.303+0000 7f0160bb7640 1 -- 172.21.15.16:0/2652805044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0148003aa0 msgr2=0x7f015c157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.303+0000 7f0160bb7640 1 --2- 172.21.15.16:0/2652805044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0148003aa0 0x7f015c157ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f014c000c00 tx=0x7f015c0d0050 comp rx=0 tx=0).stop 2024-09-12T01:32:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.304+0000 7f0160bb7640 1 -- 172.21.15.16:0/2652805044 shutdown_connections 2024-09-12T01:32:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.304+0000 7f0160bb7640 1 --2- 172.21.15.16:0/2652805044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f015c0b0b10 0x7f015c0c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.304+0000 7f0160bb7640 1 --2- 172.21.15.16:0/2652805044 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f015c0b0690 0x7f015c0c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.304+0000 7f0160bb7640 1 --2- 172.21.15.16:0/2652805044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0148003aa0 0x7f015c157ef0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.304+0000 7f0160bb7640 1 -- 172.21.15.16:0/2652805044 >> 172.21.15.16:0/2652805044 conn(0x7f015c06baf0 msgr2=0x7f015c06f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.304+0000 7f0160bb7640 1 -- 172.21.15.16:0/2652805044 shutdown_connections 2024-09-12T01:32:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:48.304+0000 7f0160bb7640 1 -- 172.21.15.16:0/2652805044 wait complete. 2024-09-12T01:32:48.314 INFO:tasks.ceph:check_enable_crimson: False 2024-09-12T01:32:48.314 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-12T01:32:48.314 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:48.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-12T01:32:48.340 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:48.340 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-12T01:32:48.342 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:48.342+0000 7fc061c21040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:32:48.343 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:48.343+0000 7fa2fe01c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:32:48.368 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:48.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-12T01:32:48.384 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:48.385+0000 7fc061c21040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:32:48.387 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:48.387+0000 7fa2fe01c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:32:48.412 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:32:48.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-12T01:32:48.477 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:48.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-12T01:32:48.503 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:48.503 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-12T01:32:48.531 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:48.531+0000 7fc061c21040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:32:48.535 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:48.536+0000 7fa2fe01c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:32:48.556 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:48.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-12T01:32:48.584 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:48.585+0000 7fc061c21040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:32:48.588 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:48.588+0000 7fa2fe01c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:32:48.609 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:32:48.609 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-12T01:32:48.675 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new a3801af9-d983-47ce-99e7-c99f040f096c 0 2024-09-12T01:32:48.755 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:48.755+0000 7fc061c21040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:32:48.758 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:48.759+0000 7fa2fe01c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:32:48.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.818+0000 7fe1beca4640 1 Processor -- start 2024-09-12T01:32:48.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.818+0000 7fe1beca4640 1 -- start start 2024-09-12T01:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.818+0000 7fe1beca4640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1b815e780 0x7fe1b80c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.819+0000 7fe1beca4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1b80c66d0 0x7fe1b80cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.819+0000 7fe1beca4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1b815eb60 con 0x7fe1b806fa30 2024-09-12T01:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.819+0000 7fe1beca4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe1b80c6b40 con 0x7fe1b80c66d0 2024-09-12T01:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.819+0000 7fe1beca4640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fe1b80b3500 con 0x7fe1b815e780 2024-09-12T01:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.819+0000 7fe1bca40640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1b815e780 0x7fe1b80c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.819+0000 7fe1bca40640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1b815e780 0x7fe1b80c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38818/0 (socket says 172.21.15.33:38818) 2024-09-12T01:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.819+0000 7fe1bca40640 1 -- 172.21.15.33:0/18609747 learned_addr learned my addr 172.21.15.33:0/18609747 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.819+0000 7fe1bd241640 1 --2- 172.21.15.33:0/18609747 >> v2:172.21.15.16:3300/0 conn(0x7fe1b80c66d0 0x7fe1b80cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.819+0000 7fe1bca40640 1 -- 172.21.15.33:0/18609747 >> v2:172.21.15.16:3300/0 conn(0x7fe1b80c66d0 msgr2=0x7fe1b80cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.819+0000 7fe1bca40640 1 --2- 172.21.15.33:0/18609747 >> v2:172.21.15.16:3300/0 conn(0x7fe1b80c66d0 0x7fe1b80cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.819+0000 7fe1bca40640 1 -- 172.21.15.33:0/18609747 >> v1:172.21.15.16:6789/0 conn(0x7fe1b806fa30 legacy=0x7fe1b80c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.819+0000 7fe1bca40640 1 -- 172.21.15.33:0/18609747 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b80c6d90 con 0x7fe1b815e780 2024-09-12T01:32:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.820+0000 7fe1bca40640 1 --2- 172.21.15.33:0/18609747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b815e780 0x7fe1b80c7b20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8071640 tx=0x7fe1ac02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ddb45e41627eae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.820+0000 7fe1b77fe640 1 -- 172.21.15.33:0/18609747 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac037070 con 0x7fe1b815e780 2024-09-12T01:32:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.820+0000 7fe1b77fe640 1 -- 172.21.15.33:0/18609747 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ac032440 con 0x7fe1b815e780 2024-09-12T01:32:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.820+0000 7fe1b77fe640 1 -- 172.21.15.33:0/18609747 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac032810 con 0x7fe1b815e780 2024-09-12T01:32:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.821+0000 7fe1beca4640 1 -- 172.21.15.33:0/18609747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b815e780 msgr2=0x7fe1b80c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.821+0000 7fe1beca4640 1 --2- 172.21.15.33:0/18609747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b815e780 0x7fe1b80c7b20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8071640 tx=0x7fe1ac02a270 comp rx=0 tx=0).stop 2024-09-12T01:32:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.821+0000 7fe1beca4640 1 -- 172.21.15.33:0/18609747 shutdown_connections 2024-09-12T01:32:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.821+0000 7fe1beca4640 1 --2- 172.21.15.33:0/18609747 >> v2:172.21.15.16:3300/0 conn(0x7fe1b80c66d0 0x7fe1b80cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.821+0000 7fe1beca4640 1 --2- 172.21.15.33:0/18609747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b815e780 0x7fe1b80c7b20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.821+0000 7fe1beca4640 1 -- 172.21.15.33:0/18609747 >> 172.21.15.33:0/18609747 conn(0x7fe1b809aa30 msgr2=0x7fe1b80b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.821+0000 7fe1beca4640 1 -- 172.21.15.33:0/18609747 shutdown_connections 2024-09-12T01:32:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.821+0000 7fe1beca4640 1 -- 172.21.15.33:0/18609747 wait complete. 2024-09-12T01:32:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.823+0000 7fe1beca4640 1 Processor -- start 2024-09-12T01:32:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.823+0000 7fe1beca4640 1 -- start start 2024-09-12T01:32:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.824+0000 7fe1beca4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b806fa30 0x7fe1b80b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.824+0000 7fe1beca4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe1b80c66d0 0x7fe1b814dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.824+0000 7fe1beca4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1b815e780 0x7fe1b81f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.824+0000 7fe1beca4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b80c7070 con 0x7fe1b815e780 2024-09-12T01:32:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.824+0000 7fe1beca4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b80b3a80 con 0x7fe1b806fa30 2024-09-12T01:32:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.824+0000 7fe1beca4640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b806fe90 con 0x7fe1b80c66d0 2024-09-12T01:32:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.824+0000 7fe1b7fff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe1b80c66d0 0x7fe1b814dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.824+0000 7fe1b7fff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe1b80c66d0 0x7fe1b814dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.33:52918/0 (socket says 172.21.15.33:52918) 2024-09-12T01:32:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.824+0000 7fe1b7fff640 1 -- 172.21.15.33:0/2886800257 learned_addr learned my addr 172.21.15.33:0/2886800257 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:32:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.824+0000 7fe1bd241640 1 --2- 172.21.15.33:0/2886800257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1b815e780 0x7fe1b81f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.824+0000 7fe1bca40640 1 --2- 172.21.15.33:0/2886800257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b806fa30 0x7fe1b80b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1bca40640 1 -- 172.21.15.33:0/2886800257 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe1b80c66d0 msgr2=0x7fe1b814dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1bca40640 1 --2- 172.21.15.33:0/2886800257 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe1b80c66d0 0x7fe1b814dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1bca40640 1 -- 172.21.15.33:0/2886800257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1b815e780 msgr2=0x7fe1b81f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1bca40640 1 --2- 172.21.15.33:0/2886800257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1b815e780 0x7fe1b81f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1bca40640 1 -- 172.21.15.33:0/2886800257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b81f6580 con 0x7fe1b806fa30 2024-09-12T01:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1b7fff640 1 --2- 172.21.15.33:0/2886800257 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe1b80c66d0 0x7fe1b814dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1bca40640 1 --2- 172.21.15.33:0/2886800257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b806fa30 0x7fe1b80b4050 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac031000 tx=0x7fe1ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1b57fa640 1 -- 172.21.15.33:0/2886800257 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac041030 con 0x7fe1b806fa30 2024-09-12T01:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1b57fa640 1 -- 172.21.15.33:0/2886800257 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ac030440 con 0x7fe1b806fa30 2024-09-12T01:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1b57fa640 1 -- 172.21.15.33:0/2886800257 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac037040 con 0x7fe1b806fa30 2024-09-12T01:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1beca4640 1 -- 172.21.15.33:0/2886800257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b80b7070 con 0x7fe1b806fa30 2024-09-12T01:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1beca4640 1 -- 172.21.15.33:0/2886800257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b80c98e0 con 0x7fe1b806fa30 2024-09-12T01:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1b57fa640 1 -- 172.21.15.33:0/2886800257 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe1ac044070 con 0x7fe1b806fa30 2024-09-12T01:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.825+0000 7fe1b57fa640 1 -- 172.21.15.33:0/2886800257 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fe1ac03da60 con 0x7fe1b806fa30 2024-09-12T01:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.826+0000 7fe1beca4640 1 -- 172.21.15.33:0/2886800257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b80c7440 con 0x7fe1b806fa30 2024-09-12T01:32:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.828+0000 7fe1b57fa640 1 -- 172.21.15.33:0/2886800257 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe1ac03dc60 con 0x7fe1b806fa30 2024-09-12T01:32:48.874 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:48.874+0000 7fa2fe01c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:32:48.875 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:48.875+0000 7fc061c21040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:32:48.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.899+0000 7fe1beca4640 1 -- 172.21.15.33:0/2886800257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a3801af9-d983-47ce-99e7-c99f040f096c", "id": 0} v 0) -- 0x7fe1b815d410 con 0x7fe1b806fa30 2024-09-12T01:32:48.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.905+0000 7fe1b57fa640 1 -- 172.21.15.33:0/2886800257 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a3801af9-d983-47ce-99e7-c99f040f096c", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fe1ac030830 con 0x7fe1b806fa30 2024-09-12T01:32:48.904 INFO:teuthology.orchestra.run.smithi033.stdout:0 2024-09-12T01:32:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.905+0000 7fe1beca4640 1 -- 172.21.15.33:0/2886800257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b806fa30 msgr2=0x7fe1b80b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.905+0000 7fe1beca4640 1 --2- 172.21.15.33:0/2886800257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b806fa30 0x7fe1b80b4050 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac031000 tx=0x7fe1ac0046c0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.906+0000 7fe1beca4640 1 -- 172.21.15.33:0/2886800257 shutdown_connections 2024-09-12T01:32:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.906+0000 7fe1beca4640 1 --2- 172.21.15.33:0/2886800257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1b815e780 0x7fe1b81f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.906+0000 7fe1beca4640 1 --2- 172.21.15.33:0/2886800257 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe1b80c66d0 0x7fe1b814dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.906+0000 7fe1beca4640 1 --2- 172.21.15.33:0/2886800257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b806fa30 0x7fe1b80b4050 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.906+0000 7fe1beca4640 1 -- 172.21.15.33:0/2886800257 >> 172.21.15.33:0/2886800257 conn(0x7fe1b809aa30 msgr2=0x7fe1b80cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.906+0000 7fe1beca4640 1 -- 172.21.15.33:0/2886800257 shutdown_connections 2024-09-12T01:32:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:48.906+0000 7fe1beca4640 1 -- 172.21.15.33:0/2886800257 wait complete. 2024-09-12T01:32:48.915 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new 8d151a65-9f20-45b0-9c37-b58c1e5748a1 1 2024-09-12T01:32:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.057+0000 7fa41358d640 1 Processor -- start 2024-09-12T01:32:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.057+0000 7fa41358d640 1 -- start start 2024-09-12T01:32:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.058+0000 7fa41358d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa40c0adfb0 0x7fa40c154a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.058+0000 7fa41358d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa40c0af5f0 con 0x7fa40c06fe80 2024-09-12T01:32:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.058+0000 7fa41358d640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa40c0af8d0 con 0x7fa40c0af010 2024-09-12T01:32:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.058+0000 7fa41358d640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fa40c0b0200 con 0x7fa40c0adfb0 2024-09-12T01:32:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.058+0000 7fa411b2a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa40c0adfb0 0x7fa40c154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.058+0000 7fa410b28640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa40c0af010 0x7fa40c0ad8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37340/0 (socket says 172.21.15.33:37340) 2024-09-12T01:32:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.058+0000 7fa410b28640 1 -- 172.21.15.33:0/2917026132 learned_addr learned my addr 172.21.15.33:0/2917026132 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-09-12T01:32:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.059+0000 7fa411b2a640 1 -- 172.21.15.33:0/2917026132 >> v1:172.21.15.33:6789/0 conn(0x7fa40c0af010 legacy=0x7fa40c0ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.059+0000 7fa411b2a640 1 -- 172.21.15.33:0/2917026132 >> v1:172.21.15.16:6789/0 conn(0x7fa40c06fe80 legacy=0x7fa40c15f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.059+0000 7fa403fff640 1 -- 172.21.15.33:0/2917026132 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1687407526 0 0) 0x7fa40c0af8d0 con 0x7fa40c0af010 2024-09-12T01:32:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.059+0000 7fa411b2a640 1 -- 172.21.15.33:0/2917026132 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c0afbb0 con 0x7fa40c0adfb0 2024-09-12T01:32:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.060+0000 7fa411b2a640 1 --2- 172.21.15.33:0/2917026132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa40c0adfb0 0x7fa40c154a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa404009d90 tx=0x7fa40402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6124be9be261bb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.060+0000 7fa403fff640 1 -- 172.21.15.33:0/2917026132 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404007740 con 0x7fa40c0adfb0 2024-09-12T01:32:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.060+0000 7fa403fff640 1 -- 172.21.15.33:0/2917026132 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4040078e0 con 0x7fa40c0adfb0 2024-09-12T01:32:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.060+0000 7fa403fff640 1 -- 172.21.15.33:0/2917026132 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404007cb0 con 0x7fa40c0adfb0 2024-09-12T01:32:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.061+0000 7fa41358d640 1 -- 172.21.15.33:0/2917026132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa40c0adfb0 msgr2=0x7fa40c154a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.061+0000 7fa41358d640 1 --2- 172.21.15.33:0/2917026132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa40c0adfb0 0x7fa40c154a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa404009d90 tx=0x7fa40402a6f0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.061+0000 7fa41358d640 1 -- 172.21.15.33:0/2917026132 shutdown_connections 2024-09-12T01:32:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.061+0000 7fa41358d640 1 --2- 172.21.15.33:0/2917026132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa40c0adfb0 0x7fa40c154a80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.061+0000 7fa41358d640 1 -- 172.21.15.33:0/2917026132 >> 172.21.15.33:0/2917026132 conn(0x7fa40c073390 msgr2=0x7fa40c06baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.061+0000 7fa41358d640 1 -- 172.21.15.33:0/2917026132 shutdown_connections 2024-09-12T01:32:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.061+0000 7fa41358d640 1 -- 172.21.15.33:0/2917026132 wait complete. 2024-09-12T01:32:49.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.063+0000 7fa41358d640 1 Processor -- start 2024-09-12T01:32:49.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.063+0000 7fa41358d640 1 -- start start 2024-09-12T01:32:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa41358d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa40c06fe80 0x7fa40c150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa41358d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa40c0adfb0 0x7fa40c150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa41358d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa40c0af010 0x7fa40c15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa41358d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c0b61f0 con 0x7fa40c0af010 2024-09-12T01:32:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa41358d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c0b6070 con 0x7fa40c06fe80 2024-09-12T01:32:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa41358d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa40c0b6370 con 0x7fa40c0adfb0 2024-09-12T01:32:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa411329640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa40c06fe80 0x7fa40c150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa410b28640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa40c0adfb0 0x7fa40c150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa411329640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa40c06fe80 0x7fa40c150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38852/0 (socket says 172.21.15.33:38852) 2024-09-12T01:32:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa411329640 1 -- 172.21.15.33:0/3790605650 learned_addr learned my addr 172.21.15.33:0/3790605650 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:32:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa410b28640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa40c0adfb0 0x7fa40c150f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.33:52924/0 (socket says 172.21.15.33:52924) 2024-09-12T01:32:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa411b2a640 1 --2- 172.21.15.33:0/3790605650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa40c0af010 0x7fa40c15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa411329640 1 -- 172.21.15.33:0/3790605650 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa40c0adfb0 msgr2=0x7fa40c150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa411329640 1 --2- 172.21.15.33:0/3790605650 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa40c0adfb0 0x7fa40c150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa411329640 1 -- 172.21.15.33:0/3790605650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa40c0af010 msgr2=0x7fa40c15b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa411329640 1 --2- 172.21.15.33:0/3790605650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa40c0af010 0x7fa40c15b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.064+0000 7fa411329640 1 -- 172.21.15.33:0/3790605650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c159810 con 0x7fa40c06fe80 2024-09-12T01:32:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.065+0000 7fa411329640 1 --2- 172.21.15.33:0/3790605650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa40c06fe80 0x7fa40c150a40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa40800c400 tx=0x7fa40800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.065+0000 7fa401ffb640 1 -- 172.21.15.33:0/3790605650 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408017070 con 0x7fa40c06fe80 2024-09-12T01:32:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.065+0000 7fa401ffb640 1 -- 172.21.15.33:0/3790605650 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4080052d0 con 0x7fa40c06fe80 2024-09-12T01:32:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.065+0000 7fa401ffb640 1 -- 172.21.15.33:0/3790605650 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408003830 con 0x7fa40c06fe80 2024-09-12T01:32:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.065+0000 7fa41358d640 1 -- 172.21.15.33:0/3790605650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c159ab0 con 0x7fa40c06fe80 2024-09-12T01:32:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.065+0000 7fa401ffb640 1 -- 172.21.15.33:0/3790605650 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa40801a070 con 0x7fa40c06fe80 2024-09-12T01:32:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.065+0000 7fa41358d640 1 -- 172.21.15.33:0/3790605650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c15a070 con 0x7fa40c06fe80 2024-09-12T01:32:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.065+0000 7fa401ffb640 1 -- 172.21.15.33:0/3790605650 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fa408011c20 con 0x7fa40c06fe80 2024-09-12T01:32:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.066+0000 7fa41358d640 1 -- 172.21.15.33:0/3790605650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c15f1e0 con 0x7fa40c06fe80 2024-09-12T01:32:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.068+0000 7fa401ffb640 1 -- 172.21.15.33:0/3790605650 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa408022070 con 0x7fa40c06fe80 2024-09-12T01:32:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.139+0000 7fa41358d640 1 -- 172.21.15.33:0/3790605650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8d151a65-9f20-45b0-9c37-b58c1e5748a1", "id": 1} v 0) -- 0x7fa40c0adc30 con 0x7fa40c06fe80 2024-09-12T01:32:49.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.142+0000 7fa401ffb640 1 -- 172.21.15.33:0/3790605650 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8d151a65-9f20-45b0-9c37-b58c1e5748a1", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fa408011e20 con 0x7fa40c06fe80 2024-09-12T01:32:49.141 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-09-12T01:32:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.143+0000 7fa41358d640 1 -- 172.21.15.33:0/3790605650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa40c06fe80 msgr2=0x7fa40c150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.143+0000 7fa41358d640 1 --2- 172.21.15.33:0/3790605650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa40c06fe80 0x7fa40c150a40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa40800c400 tx=0x7fa40800c8c0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.143+0000 7fa41358d640 1 -- 172.21.15.33:0/3790605650 shutdown_connections 2024-09-12T01:32:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.143+0000 7fa41358d640 1 --2- 172.21.15.33:0/3790605650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa40c0af010 0x7fa40c15b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.143+0000 7fa41358d640 1 --2- 172.21.15.33:0/3790605650 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa40c0adfb0 0x7fa40c150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.143+0000 7fa41358d640 1 --2- 172.21.15.33:0/3790605650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa40c06fe80 0x7fa40c150a40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.143+0000 7fa41358d640 1 -- 172.21.15.33:0/3790605650 >> 172.21.15.33:0/3790605650 conn(0x7fa40c073390 msgr2=0x7fa40c0744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.143+0000 7fa41358d640 1 -- 172.21.15.33:0/3790605650 shutdown_connections 2024-09-12T01:32:49.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.143+0000 7fa41358d640 1 -- 172.21.15.33:0/3790605650 wait complete. 2024-09-12T01:32:49.152 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new 6c1c9bd6-738d-45e1-a96c-55d33076e777 2 2024-09-12T01:32:49.218 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:49.218+0000 7fa2fe01c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:32:49.234 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:49.235+0000 7fc061c21040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:32:49.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.256+0000 7f84b7ee6640 1 Processor -- start 2024-09-12T01:32:49.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.256+0000 7f84b7ee6640 1 -- start start 2024-09-12T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.257+0000 7f84b7ee6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84b015b0b0 0x7f84b0072ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.257+0000 7f84b7ee6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f84b0073c30 0x7f84b015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.257+0000 7f84b7ee6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b0073370 con 0x7f84b006faf0 2024-09-12T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.257+0000 7f84b7ee6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f84b015bb50 con 0x7f84b015b0b0 2024-09-12T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.257+0000 7f84b7ee6640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f84b0073650 con 0x7f84b0073c30 2024-09-12T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.257+0000 7f84b5481640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f84b0073c30 0x7f84b015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.257+0000 7f84b5481640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f84b0073c30 0x7f84b015bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38864/0 (socket says 172.21.15.33:38864) 2024-09-12T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.257+0000 7f84b5481640 1 -- 172.21.15.33:0/2747449723 learned_addr learned my addr 172.21.15.33:0/2747449723 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.257+0000 7f84b5c82640 1 --2- 172.21.15.33:0/2747449723 >> v2:172.21.15.16:3300/0 conn(0x7f84b015b0b0 0x7f84b0072ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.257+0000 7f84b5481640 1 -- 172.21.15.33:0/2747449723 >> v2:172.21.15.16:3300/0 conn(0x7f84b015b0b0 msgr2=0x7f84b0072ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.257+0000 7f84b5481640 1 --2- 172.21.15.33:0/2747449723 >> v2:172.21.15.16:3300/0 conn(0x7f84b015b0b0 0x7f84b0072ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.257+0000 7f84b5481640 1 -- 172.21.15.33:0/2747449723 >> v1:172.21.15.16:6789/0 conn(0x7f84b006faf0 legacy=0x7f84b00aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.258+0000 7f84b5481640 1 -- 172.21.15.33:0/2747449723 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b015b7e0 con 0x7f84b0073c30 2024-09-12T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.258+0000 7f84b5481640 1 --2- 172.21.15.33:0/2747449723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b0073c30 0x7f84b015bfe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f84b00ae9c0 tx=0x7f84ac02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a77bb78e38b37208 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.258+0000 7f84b4c80640 1 -- 172.21.15.33:0/2747449723 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac037070 con 0x7f84b0073c30 2024-09-12T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.259+0000 7f84b4c80640 1 -- 172.21.15.33:0/2747449723 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84ac032440 con 0x7f84b0073c30 2024-09-12T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.259+0000 7f84b4c80640 1 -- 172.21.15.33:0/2747449723 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac032810 con 0x7f84b0073c30 2024-09-12T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.259+0000 7f84b7ee6640 1 -- 172.21.15.33:0/2747449723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b0073c30 msgr2=0x7f84b015bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.259+0000 7f84b7ee6640 1 --2- 172.21.15.33:0/2747449723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b0073c30 0x7f84b015bfe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f84b00ae9c0 tx=0x7f84ac02a270 comp rx=0 tx=0).stop 2024-09-12T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.259+0000 7f84b7ee6640 1 -- 172.21.15.33:0/2747449723 shutdown_connections 2024-09-12T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.259+0000 7f84b7ee6640 1 --2- 172.21.15.33:0/2747449723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b0073c30 0x7f84b015bfe0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.259+0000 7f84b7ee6640 1 --2- 172.21.15.33:0/2747449723 >> v2:172.21.15.16:3300/0 conn(0x7f84b015b0b0 0x7f84b0072ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.259+0000 7f84b7ee6640 1 -- 172.21.15.33:0/2747449723 >> 172.21.15.33:0/2747449723 conn(0x7f84b00b2c50 msgr2=0x7f84b00b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.259+0000 7f84b7ee6640 1 -- 172.21.15.33:0/2747449723 shutdown_connections 2024-09-12T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.259+0000 7f84b7ee6640 1 -- 172.21.15.33:0/2747449723 wait complete. 2024-09-12T01:32:49.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.260+0000 7f84b7ee6640 1 Processor -- start 2024-09-12T01:32:49.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.260+0000 7f84b7ee6640 1 -- start start 2024-09-12T01:32:49.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.261+0000 7f84b7ee6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b006faf0 0x7f84b0146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.261+0000 7f84b7ee6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b0073c30 0x7f84b0147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.261+0000 7f84b7ee6640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84b015b0b0 0x7f84b015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.261+0000 7f84b7ee6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84b006ff80 con 0x7f84b006faf0 2024-09-12T01:32:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.261+0000 7f84b7ee6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84b015b490 con 0x7f84b0073c30 2024-09-12T01:32:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.261+0000 7f84b7ee6640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f84b0070100 con 0x7f84b015b0b0 2024-09-12T01:32:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.261+0000 7f84b5481640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b0073c30 0x7f84b0147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.261+0000 7f84b5481640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b0073c30 0x7f84b0147370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38870/0 (socket says 172.21.15.33:38870) 2024-09-12T01:32:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.261+0000 7f84b5481640 1 -- 172.21.15.33:0/543824143 learned_addr learned my addr 172.21.15.33:0/543824143 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:32:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.261+0000 7f84b6483640 1 --2- 172.21.15.33:0/543824143 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84b015b0b0 0x7f84b015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.261+0000 7f84b5c82640 1 --2- 172.21.15.33:0/543824143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b006faf0 0x7f84b0146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.262+0000 7f84b5481640 1 -- 172.21.15.33:0/543824143 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84b015b0b0 msgr2=0x7f84b015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.262+0000 7f84b5481640 1 --2- 172.21.15.33:0/543824143 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84b015b0b0 0x7f84b015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.262+0000 7f84b5481640 1 -- 172.21.15.33:0/543824143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b006faf0 msgr2=0x7f84b0146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.262+0000 7f84b5481640 1 --2- 172.21.15.33:0/543824143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b006faf0 0x7f84b0146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.262+0000 7f84b5481640 1 -- 172.21.15.33:0/543824143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b0147090 con 0x7f84b0073c30 2024-09-12T01:32:49.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.262+0000 7f84b5481640 1 --2- 172.21.15.33:0/543824143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b0073c30 0x7f84b0147370 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f84ac000c00 tx=0x7f84ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:49.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.262+0000 7f84a27fc640 1 -- 172.21.15.33:0/543824143 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac041070 con 0x7f84b0073c30 2024-09-12T01:32:49.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.262+0000 7f84a27fc640 1 -- 172.21.15.33:0/543824143 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84ac03c870 con 0x7f84b0073c30 2024-09-12T01:32:49.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.262+0000 7f84b7ee6640 1 -- 172.21.15.33:0/543824143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b00cdf90 con 0x7f84b0073c30 2024-09-12T01:32:49.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.262+0000 7f84a27fc640 1 -- 172.21.15.33:0/543824143 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac037040 con 0x7f84b0073c30 2024-09-12T01:32:49.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.262+0000 7f84b7ee6640 1 -- 172.21.15.33:0/543824143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b00ce550 con 0x7f84b0073c30 2024-09-12T01:32:49.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.262+0000 7f84a27fc640 1 -- 172.21.15.33:0/543824143 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f84ac044070 con 0x7f84b0073c30 2024-09-12T01:32:49.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.262+0000 7f84a27fc640 1 -- 172.21.15.33:0/543824143 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f84ac03d3f0 con 0x7f84b0073c30 2024-09-12T01:32:49.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.263+0000 7f84b7ee6640 1 -- 172.21.15.33:0/543824143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b0072ce0 con 0x7f84b0073c30 2024-09-12T01:32:49.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.265+0000 7f84a27fc640 1 -- 172.21.15.33:0/543824143 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f84b0147090 con 0x7f84b0073c30 2024-09-12T01:32:49.273 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:49.274+0000 7fa2fe01c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:32:49.294 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:49.294+0000 7fc061c21040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:32:49.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.336+0000 7f84b7ee6640 1 -- 172.21.15.33:0/543824143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6c1c9bd6-738d-45e1-a96c-55d33076e777", "id": 2} v 0) -- 0x7f84b015f720 con 0x7f84b0073c30 2024-09-12T01:32:49.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.339+0000 7f84a27fc640 1 -- 172.21.15.33:0/543824143 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6c1c9bd6-738d-45e1-a96c-55d33076e777", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f84ac046450 con 0x7f84b0073c30 2024-09-12T01:32:49.339 INFO:teuthology.orchestra.run.smithi033.stdout:2 2024-09-12T01:32:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.341+0000 7f84b7ee6640 1 -- 172.21.15.33:0/543824143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b0073c30 msgr2=0x7f84b0147370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.341+0000 7f84b7ee6640 1 --2- 172.21.15.33:0/543824143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b0073c30 0x7f84b0147370 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f84ac000c00 tx=0x7f84ac0046c0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.341+0000 7f84b7ee6640 1 -- 172.21.15.33:0/543824143 shutdown_connections 2024-09-12T01:32:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.341+0000 7f84b7ee6640 1 --2- 172.21.15.33:0/543824143 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84b015b0b0 0x7f84b015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.341+0000 7f84b7ee6640 1 --2- 172.21.15.33:0/543824143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b0073c30 0x7f84b0147370 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.341+0000 7f84b7ee6640 1 --2- 172.21.15.33:0/543824143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b006faf0 0x7f84b0146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.341+0000 7f84b7ee6640 1 -- 172.21.15.33:0/543824143 >> 172.21.15.33:0/543824143 conn(0x7f84b00b2c50 msgr2=0x7f84b015e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.341+0000 7f84b7ee6640 1 -- 172.21.15.33:0/543824143 shutdown_connections 2024-09-12T01:32:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.341+0000 7f84b7ee6640 1 -- 172.21.15.33:0/543824143 wait complete. 2024-09-12T01:32:49.350 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new 1acf626b-5963-49ef-8a2a-bb551c666d33 3 2024-09-12T01:32:49.431 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:49.431+0000 7fa2fe01c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:32:49.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.452+0000 7fb1a2027640 1 Processor -- start 2024-09-12T01:32:49.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.452+0000 7fb1a2027640 1 -- start start 2024-09-12T01:32:49.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.453+0000 7fb1a2027640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb19c15c2e0 0x7fb19c0d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.453+0000 7fb1a2027640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb19c15fd80 0x7fb19c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.453+0000 7fb1a2027640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb19c160340 con 0x7fb19c0cebb0 2024-09-12T01:32:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.453+0000 7fb1a2027640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb19c0d02a0 con 0x7fb19c15fd80 2024-09-12T01:32:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.453+0000 7fb1a2027640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb19c160620 con 0x7fb19c15c2e0 2024-09-12T01:32:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.453+0000 7fb19b7fe640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb19c15c2e0 0x7fb19c0d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.453+0000 7fb19affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb19c15fd80 0x7fb19c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.453+0000 7fb19b7fe640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb19c15c2e0 0x7fb19c0d19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.33:52946/0 (socket says 172.21.15.33:52946) 2024-09-12T01:32:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.453+0000 7fb19affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb19c15fd80 0x7fb19c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38872/0 (socket says 172.21.15.33:38872) 2024-09-12T01:32:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.453+0000 7fb19b7fe640 1 -- 172.21.15.33:0/2657673653 learned_addr learned my addr 172.21.15.33:0/2657673653 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:32:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.454+0000 7fb19affd640 1 -- 172.21.15.33:0/2657673653 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb19c15c2e0 msgr2=0x7fb19c0d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.454+0000 7fb19affd640 1 --2- 172.21.15.33:0/2657673653 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb19c15c2e0 0x7fb19c0d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.454+0000 7fb19affd640 1 -- 172.21.15.33:0/2657673653 >> v1:172.21.15.16:6789/0 conn(0x7fb19c0cebb0 legacy=0x7fb19c0d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.454+0000 7fb19affd640 1 -- 172.21.15.33:0/2657673653 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19c160900 con 0x7fb19c15fd80 2024-09-12T01:32:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.454+0000 7fb19affd640 1 --2- 172.21.15.33:0/2657673653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c15fd80 0x7fb19c0ce670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb1900097a0 tx=0x7fb19002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1884cf5e8c81a79e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:49.455 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:49.454+0000 7fc061c21040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:32:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.455+0000 7fb19a7fc640 1 -- 172.21.15.33:0/2657673653 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190037070 con 0x7fb19c15fd80 2024-09-12T01:32:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.455+0000 7fb19a7fc640 1 -- 172.21.15.33:0/2657673653 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb190032440 con 0x7fb19c15fd80 2024-09-12T01:32:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.455+0000 7fb19a7fc640 1 -- 172.21.15.33:0/2657673653 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190032810 con 0x7fb19c15fd80 2024-09-12T01:32:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.455+0000 7fb1a2027640 1 -- 172.21.15.33:0/2657673653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c15fd80 msgr2=0x7fb19c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.455+0000 7fb1a2027640 1 --2- 172.21.15.33:0/2657673653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c15fd80 0x7fb19c0ce670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb1900097a0 tx=0x7fb19002a180 comp rx=0 tx=0).stop 2024-09-12T01:32:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.455+0000 7fb1a2027640 1 -- 172.21.15.33:0/2657673653 shutdown_connections 2024-09-12T01:32:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.455+0000 7fb1a2027640 1 --2- 172.21.15.33:0/2657673653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c15fd80 0x7fb19c0ce670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.455+0000 7fb1a2027640 1 --2- 172.21.15.33:0/2657673653 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb19c15c2e0 0x7fb19c0d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.455+0000 7fb1a2027640 1 -- 172.21.15.33:0/2657673653 >> 172.21.15.33:0/2657673653 conn(0x7fb19c071830 msgr2=0x7fb19c071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.455+0000 7fb1a2027640 1 -- 172.21.15.33:0/2657673653 shutdown_connections 2024-09-12T01:32:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.455+0000 7fb1a2027640 1 -- 172.21.15.33:0/2657673653 wait complete. 2024-09-12T01:32:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.456+0000 7fb1a2027640 1 Processor -- start 2024-09-12T01:32:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.457+0000 7fb1a2027640 1 -- start start 2024-09-12T01:32:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.457+0000 7fb1a2027640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0cebb0 0x7fb19c146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.457+0000 7fb1a2027640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb19c15c2e0 0x7fb19c152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.457+0000 7fb1a2027640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb19c15fd80 0x7fb19c156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.457+0000 7fb1a2027640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c0d2850 con 0x7fb19c15c2e0 2024-09-12T01:32:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.457+0000 7fb1a2027640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c0d1ee0 con 0x7fb19c0cebb0 2024-09-12T01:32:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.457+0000 7fb1a2027640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb19c0d29d0 con 0x7fb19c15fd80 2024-09-12T01:32:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb19b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0cebb0 0x7fb19c146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb19b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0cebb0 0x7fb19c146530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38882/0 (socket says 172.21.15.33:38882) 2024-09-12T01:32:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb19b7fe640 1 -- 172.21.15.33:0/3810990583 learned_addr learned my addr 172.21.15.33:0/3810990583 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:32:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb19bfff640 1 --2- 172.21.15.33:0/3810990583 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb19c15fd80 0x7fb19c156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb19affd640 1 --2- 172.21.15.33:0/3810990583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb19c15c2e0 0x7fb19c152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb19b7fe640 1 -- 172.21.15.33:0/3810990583 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb19c15fd80 msgr2=0x7fb19c156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb19b7fe640 1 --2- 172.21.15.33:0/3810990583 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb19c15fd80 0x7fb19c156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb19b7fe640 1 -- 172.21.15.33:0/3810990583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb19c15c2e0 msgr2=0x7fb19c152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb19b7fe640 1 --2- 172.21.15.33:0/3810990583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb19c15c2e0 0x7fb19c152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb19b7fe640 1 -- 172.21.15.33:0/3810990583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19c0b50c0 con 0x7fb19c0cebb0 2024-09-12T01:32:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb19b7fe640 1 --2- 172.21.15.33:0/3810990583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0cebb0 0x7fb19c146530 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009fd0 tx=0x7fb18c00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb183fff640 1 -- 172.21.15.33:0/3810990583 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c017070 con 0x7fb19c0cebb0 2024-09-12T01:32:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb183fff640 1 -- 172.21.15.33:0/3810990583 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb18c00c940 con 0x7fb19c0cebb0 2024-09-12T01:32:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb1a2027640 1 -- 172.21.15.33:0/3810990583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19c155e10 con 0x7fb19c0cebb0 2024-09-12T01:32:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb183fff640 1 -- 172.21.15.33:0/3810990583 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c00cd10 con 0x7fb19c0cebb0 2024-09-12T01:32:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.458+0000 7fb1a2027640 1 -- 172.21.15.33:0/3810990583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb19c0b5660 con 0x7fb19c0cebb0 2024-09-12T01:32:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.459+0000 7fb183fff640 1 -- 172.21.15.33:0/3810990583 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb18c01a070 con 0x7fb19c0cebb0 2024-09-12T01:32:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.459+0000 7fb183fff640 1 -- 172.21.15.33:0/3810990583 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fb18c01c8e0 con 0x7fb19c0cebb0 2024-09-12T01:32:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.459+0000 7fb1a2027640 1 -- 172.21.15.33:0/3810990583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb19c0d19a0 con 0x7fb19c0cebb0 2024-09-12T01:32:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.463+0000 7fb183fff640 1 -- 172.21.15.33:0/3810990583 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb18c01cb80 con 0x7fb19c0cebb0 2024-09-12T01:32:49.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.534+0000 7fb1a2027640 1 -- 172.21.15.33:0/3810990583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1acf626b-5963-49ef-8a2a-bb551c666d33", "id": 3} v 0) -- 0x7fb19c0690c0 con 0x7fb19c0cebb0 2024-09-12T01:32:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.537+0000 7fb183fff640 1 -- 172.21.15.33:0/3810990583 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1acf626b-5963-49ef-8a2a-bb551c666d33", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fb18c01c450 con 0x7fb19c0cebb0 2024-09-12T01:32:49.537 INFO:teuthology.orchestra.run.smithi033.stdout:3 2024-09-12T01:32:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.538+0000 7fb1a2027640 1 -- 172.21.15.33:0/3810990583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0cebb0 msgr2=0x7fb19c146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.538+0000 7fb1a2027640 1 --2- 172.21.15.33:0/3810990583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0cebb0 0x7fb19c146530 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009fd0 tx=0x7fb18c00ed40 comp rx=0 tx=0).stop 2024-09-12T01:32:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.538+0000 7fb1a2027640 1 -- 172.21.15.33:0/3810990583 shutdown_connections 2024-09-12T01:32:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.538+0000 7fb1a2027640 1 --2- 172.21.15.33:0/3810990583 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb19c15fd80 0x7fb19c156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.538+0000 7fb1a2027640 1 --2- 172.21.15.33:0/3810990583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb19c15c2e0 0x7fb19c152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.538+0000 7fb1a2027640 1 --2- 172.21.15.33:0/3810990583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0cebb0 0x7fb19c146530 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.538+0000 7fb1a2027640 1 -- 172.21.15.33:0/3810990583 >> 172.21.15.33:0/3810990583 conn(0x7fb19c071830 msgr2=0x7fb19c15ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.538+0000 7fb1a2027640 1 -- 172.21.15.33:0/3810990583 shutdown_connections 2024-09-12T01:32:49.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.538+0000 7fb1a2027640 1 -- 172.21.15.33:0/3810990583 wait complete. 2024-09-12T01:32:49.548 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new 79396361-0fb0-4aac-a53e-1a532d30243a 4 2024-09-12T01:32:49.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.650+0000 7f21bb998640 1 Processor -- start 2024-09-12T01:32:49.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.650+0000 7f21bb998640 1 -- start start 2024-09-12T01:32:49.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.651+0000 7f21bb998640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21b415e100 0x7f21b40b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.651+0000 7f21bb998640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21b406d210 0x7f21b40b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.651+0000 7f21bb998640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21b406ec40 con 0x7f21b40ae8e0 2024-09-12T01:32:49.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.651+0000 7f21bb998640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f21b406ef20 con 0x7f21b406d210 2024-09-12T01:32:49.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.651+0000 7f21bb998640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f21b406f200 con 0x7f21b415e100 2024-09-12T01:32:49.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.651+0000 7f21b9734640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21b415e100 0x7f21b40b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.651+0000 7f21b8f33640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f21b40ae8e0 0x7f21b406cb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37348/0 (socket says 172.21.15.33:37348) 2024-09-12T01:32:49.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.651+0000 7f21b8f33640 1 -- 172.21.15.33:0/3945628814 learned_addr learned my addr 172.21.15.33:0/3945628814 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-09-12T01:32:49.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.651+0000 7f21b9f35640 1 --2- 172.21.15.33:0/3945628814 >> v2:172.21.15.16:3300/0 conn(0x7f21b406d210 0x7f21b40b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.651+0000 7f21b3fff640 1 -- 172.21.15.33:0/3945628814 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1490060669 0 0) 0x7f21b406ec40 con 0x7f21b40ae8e0 2024-09-12T01:32:49.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.651+0000 7f21b9734640 1 -- 172.21.15.33:0/3945628814 >> v2:172.21.15.16:3300/0 conn(0x7f21b406d210 msgr2=0x7f21b40b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:49.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.651+0000 7f21b9734640 1 --2- 172.21.15.33:0/3945628814 >> v2:172.21.15.16:3300/0 conn(0x7f21b406d210 0x7f21b40b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.651+0000 7f21b9734640 1 -- 172.21.15.33:0/3945628814 >> v1:172.21.15.33:6789/0 conn(0x7f21b40ae8e0 legacy=0x7f21b406cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.652+0000 7f21b9734640 1 -- 172.21.15.33:0/3945628814 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b406f4e0 con 0x7f21b415e100 2024-09-12T01:32:49.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.652+0000 7f21b9734640 1 --2- 172.21.15.33:0/3945628814 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21b415e100 0x7f21b40b59c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0097a0 tx=0x7f21ac02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c004c49e7201051f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:49.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.652+0000 7f21b3fff640 1 -- 172.21.15.33:0/3945628814 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac037070 con 0x7f21b415e100 2024-09-12T01:32:49.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.652+0000 7f21b3fff640 1 -- 172.21.15.33:0/3945628814 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ac032440 con 0x7f21b415e100 2024-09-12T01:32:49.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.652+0000 7f21b3fff640 1 -- 172.21.15.33:0/3945628814 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac032810 con 0x7f21b415e100 2024-09-12T01:32:49.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.653+0000 7f21bb998640 1 -- 172.21.15.33:0/3945628814 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21b415e100 msgr2=0x7f21b40b59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.653+0000 7f21bb998640 1 --2- 172.21.15.33:0/3945628814 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21b415e100 0x7f21b40b59c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0097a0 tx=0x7f21ac02a180 comp rx=0 tx=0).stop 2024-09-12T01:32:49.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.653+0000 7f21bb998640 1 -- 172.21.15.33:0/3945628814 shutdown_connections 2024-09-12T01:32:49.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.653+0000 7f21bb998640 1 --2- 172.21.15.33:0/3945628814 >> v2:172.21.15.16:3300/0 conn(0x7f21b406d210 0x7f21b40b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.653+0000 7f21bb998640 1 --2- 172.21.15.33:0/3945628814 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21b415e100 0x7f21b40b59c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.653+0000 7f21bb998640 1 -- 172.21.15.33:0/3945628814 >> 172.21.15.33:0/3945628814 conn(0x7f21b4072430 msgr2=0x7f21b4072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:49.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.653+0000 7f21bb998640 1 -- 172.21.15.33:0/3945628814 shutdown_connections 2024-09-12T01:32:49.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.653+0000 7f21bb998640 1 -- 172.21.15.33:0/3945628814 wait complete. 2024-09-12T01:32:49.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.654+0000 7f21bb998640 1 Processor -- start 2024-09-12T01:32:49.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.654+0000 7f21bb998640 1 -- start start 2024-09-12T01:32:49.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.655+0000 7f21bb998640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b406d210 0x7f21b4140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.655+0000 7f21bb998640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21b40ae8e0 0x7f21b4150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.655+0000 7f21bb998640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21b415e100 0x7f21b4151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.655+0000 7f21bb998640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21b406e850 con 0x7f21b406d210 2024-09-12T01:32:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.655+0000 7f21bb998640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f21b406e6d0 con 0x7f21b415e100 2024-09-12T01:32:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.655+0000 7f21bb998640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f21b406e9d0 con 0x7f21b40ae8e0 2024-09-12T01:32:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.655+0000 7f21b8f33640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21b40ae8e0 0x7f21b4150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.655+0000 7f21b8f33640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21b40ae8e0 0x7f21b4150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.33:52960/0 (socket says 172.21.15.33:52960) 2024-09-12T01:32:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.655+0000 7f21b8f33640 1 -- 172.21.15.33:0/4193645339 learned_addr learned my addr 172.21.15.33:0/4193645339 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:32:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.655+0000 7f21b9734640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b406d210 0x7f21b4140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.655+0000 7f21b9f35640 1 --2- 172.21.15.33:0/4193645339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21b415e100 0x7f21b4151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21b8f33640 1 -- 172.21.15.33:0/4193645339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21b415e100 msgr2=0x7f21b4151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21b8f33640 1 --2- 172.21.15.33:0/4193645339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21b415e100 0x7f21b4151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21b8f33640 1 -- 172.21.15.33:0/4193645339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b406d210 msgr2=0x7f21b4140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21b8f33640 1 --2- 172.21.15.33:0/4193645339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b406d210 0x7f21b4140290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21b8f33640 1 -- 172.21.15.33:0/4193645339 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b415f4a0 con 0x7f21b40ae8e0 2024-09-12T01:32:49.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21b9734640 1 --2- 172.21.15.33:0/4193645339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b406d210 0x7f21b4140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:49.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21b9f35640 1 --2- 172.21.15.33:0/4193645339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21b415e100 0x7f21b4151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:49.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21b8f33640 1 --2- 172.21.15.33:0/4193645339 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21b40ae8e0 0x7f21b4150c50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f21a800c8c0 tx=0x7f21a800cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:49.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21b1ffb640 1 -- 172.21.15.33:0/4193645339 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8018070 con 0x7f21b40ae8e0 2024-09-12T01:32:49.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21b1ffb640 1 -- 172.21.15.33:0/4193645339 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a800a640 con 0x7f21b40ae8e0 2024-09-12T01:32:49.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21bb998640 1 -- 172.21.15.33:0/4193645339 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b415f780 con 0x7f21b40ae8e0 2024-09-12T01:32:49.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21b1ffb640 1 -- 172.21.15.33:0/4193645339 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a800aa70 con 0x7f21b40ae8e0 2024-09-12T01:32:49.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21bb998640 1 -- 172.21.15.33:0/4193645339 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b415fd40 con 0x7f21b40ae8e0 2024-09-12T01:32:49.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21b1ffb640 1 -- 172.21.15.33:0/4193645339 <== mon.2 v2:172.21.15.33:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f21a800ac10 con 0x7f21b40ae8e0 2024-09-12T01:32:49.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.656+0000 7f21b1ffb640 1 -- 172.21.15.33:0/4193645339 <== mon.2 v2:172.21.15.33:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f21a80075e0 con 0x7f21b40ae8e0 2024-09-12T01:32:49.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.657+0000 7f21bb998640 1 -- 172.21.15.33:0/4193645339 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b415ff80 con 0x7f21b40ae8e0 2024-09-12T01:32:49.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.659+0000 7f21b1ffb640 1 -- 172.21.15.33:0/4193645339 <== mon.2 v2:172.21.15.33:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f21b415ff80 con 0x7f21b40ae8e0 2024-09-12T01:32:49.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.730+0000 7f21bb998640 1 -- 172.21.15.33:0/4193645339 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "osd new", "uuid": "79396361-0fb0-4aac-a53e-1a532d30243a", "id": 4} v 0) -- 0x7f21b415e4e0 con 0x7f21b40ae8e0 2024-09-12T01:32:49.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.733+0000 7f21b1ffb640 1 -- 172.21.15.33:0/4193645339 <== mon.2 v2:172.21.15.33:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "79396361-0fb0-4aac-a53e-1a532d30243a", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f21a80077e0 con 0x7f21b40ae8e0 2024-09-12T01:32:49.732 INFO:teuthology.orchestra.run.smithi033.stdout:4 2024-09-12T01:32:49.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.734+0000 7f21bb998640 1 -- 172.21.15.33:0/4193645339 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21b40ae8e0 msgr2=0x7f21b4150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.734+0000 7f21bb998640 1 --2- 172.21.15.33:0/4193645339 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21b40ae8e0 0x7f21b4150c50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f21a800c8c0 tx=0x7f21a800cd80 comp rx=0 tx=0).stop 2024-09-12T01:32:49.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.734+0000 7f21bb998640 1 -- 172.21.15.33:0/4193645339 shutdown_connections 2024-09-12T01:32:49.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.734+0000 7f21bb998640 1 --2- 172.21.15.33:0/4193645339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21b415e100 0x7f21b4151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.734+0000 7f21bb998640 1 --2- 172.21.15.33:0/4193645339 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21b40ae8e0 0x7f21b4150c50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.734+0000 7f21bb998640 1 --2- 172.21.15.33:0/4193645339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b406d210 0x7f21b4140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.734+0000 7f21bb998640 1 -- 172.21.15.33:0/4193645339 >> 172.21.15.33:0/4193645339 conn(0x7f21b4072430 msgr2=0x7f21b40b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:49.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.734+0000 7f21bb998640 1 -- 172.21.15.33:0/4193645339 shutdown_connections 2024-09-12T01:32:49.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.734+0000 7f21bb998640 1 -- 172.21.15.33:0/4193645339 wait complete. 2024-09-12T01:32:49.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new b4779b5c-91a9-4a60-a588-39dad1b2b355 5 2024-09-12T01:32:49.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.857+0000 7f73741fd640 1 Processor -- start 2024-09-12T01:32:49.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.858+0000 7f73741fd640 1 -- start start 2024-09-12T01:32:49.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.858+0000 7f73741fd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f736c073c30 0x7f736c072ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.859+0000 7f73741fd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f736c073370 con 0x7f736c070200 2024-09-12T01:32:49.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.859+0000 7f73741fd640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f736c073650 con 0x7f736c15b0b0 2024-09-12T01:32:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.859+0000 7f73741fd640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f736c15bb50 con 0x7f736c073c30 2024-09-12T01:32:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.859+0000 7f737279a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f736c073c30 0x7f736c072ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.859+0000 7f7371f99640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f736c15b0b0 0x7f736c06faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37362/0 (socket says 172.21.15.33:37362) 2024-09-12T01:32:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.859+0000 7f7371f99640 1 -- 172.21.15.33:0/2747978196 learned_addr learned my addr 172.21.15.33:0/2747978196 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-09-12T01:32:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.859+0000 7f737279a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f736c073c30 0x7f736c072ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38904/0 (socket says 172.21.15.33:38904) 2024-09-12T01:32:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.859+0000 7f737279a640 1 -- 172.21.15.33:0/2747978196 >> v1:172.21.15.33:6789/0 conn(0x7f736c15b0b0 legacy=0x7f736c06faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.859+0000 7f737279a640 1 -- 172.21.15.33:0/2747978196 >> v1:172.21.15.16:6789/0 conn(0x7f736c070200 legacy=0x7f736c0aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.859+0000 7f737279a640 1 -- 172.21.15.33:0/2747978196 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736c15b7e0 con 0x7f736c073c30 2024-09-12T01:32:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.860+0000 7f737279a640 1 --2- 172.21.15.33:0/2747978196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c073c30 0x7f736c072ce0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7364009d90 tx=0x7f736402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9564baa8baa4d4a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.860+0000 7f7370f97640 1 -- 172.21.15.33:0/2747978196 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364007740 con 0x7f736c073c30 2024-09-12T01:32:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.860+0000 7f7370f97640 1 -- 172.21.15.33:0/2747978196 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73640078e0 con 0x7f736c073c30 2024-09-12T01:32:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.860+0000 7f7370f97640 1 -- 172.21.15.33:0/2747978196 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364007cb0 con 0x7f736c073c30 2024-09-12T01:32:49.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.861+0000 7f73741fd640 1 -- 172.21.15.33:0/2747978196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c073c30 msgr2=0x7f736c072ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.861+0000 7f73741fd640 1 --2- 172.21.15.33:0/2747978196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c073c30 0x7f736c072ce0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7364009d90 tx=0x7f736402a6f0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.861+0000 7f73741fd640 1 -- 172.21.15.33:0/2747978196 shutdown_connections 2024-09-12T01:32:49.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.861+0000 7f73741fd640 1 --2- 172.21.15.33:0/2747978196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c073c30 0x7f736c072ce0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.861+0000 7f73741fd640 1 -- 172.21.15.33:0/2747978196 >> 172.21.15.33:0/2747978196 conn(0x7f736c0b2c50 msgr2=0x7f736c0b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:49.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.861+0000 7f73741fd640 1 -- 172.21.15.33:0/2747978196 shutdown_connections 2024-09-12T01:32:49.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.861+0000 7f73741fd640 1 -- 172.21.15.33:0/2747978196 wait complete. 2024-09-12T01:32:49.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.862+0000 7f73741fd640 1 Processor -- start 2024-09-12T01:32:49.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.862+0000 7f73741fd640 1 -- start start 2024-09-12T01:32:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.863+0000 7f73741fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c070200 0x7f736c0cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.863+0000 7f73741fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f736c073c30 0x7f736c1e4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.863+0000 7f73741fd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f736c15b0b0 0x7f736c152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.863+0000 7f73741fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f736c15bfe0 con 0x7f736c073c30 2024-09-12T01:32:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.863+0000 7f73741fd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f736c15b490 con 0x7f736c070200 2024-09-12T01:32:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.863+0000 7f73741fd640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f736c15c160 con 0x7f736c15b0b0 2024-09-12T01:32:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.863+0000 7f7371f99640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c070200 0x7f736c0cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.863+0000 7f7371f99640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c070200 0x7f736c0cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38910/0 (socket says 172.21.15.33:38910) 2024-09-12T01:32:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.863+0000 7f7371f99640 1 -- 172.21.15.33:0/4094586038 learned_addr learned my addr 172.21.15.33:0/4094586038 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.863+0000 7f737279a640 1 --2- 172.21.15.33:0/4094586038 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f736c15b0b0 0x7f736c152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.863+0000 7f7371798640 1 --2- 172.21.15.33:0/4094586038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f736c073c30 0x7f736c1e4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.864+0000 7f7371f99640 1 -- 172.21.15.33:0/4094586038 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f736c15b0b0 msgr2=0x7f736c152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.864+0000 7f7371f99640 1 --2- 172.21.15.33:0/4094586038 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f736c15b0b0 0x7f736c152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.864+0000 7f7371f99640 1 -- 172.21.15.33:0/4094586038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f736c073c30 msgr2=0x7f736c1e4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.864+0000 7f7371f99640 1 --2- 172.21.15.33:0/4094586038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f736c073c30 0x7f736c1e4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.864+0000 7f7371f99640 1 -- 172.21.15.33:0/4094586038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736c1e50a0 con 0x7f736c070200 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.864+0000 7f7371f99640 1 --2- 172.21.15.33:0/4094586038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c070200 0x7f736c0cfd10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f736800c4b0 tx=0x7f736800c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.864+0000 7f735e7fc640 1 -- 172.21.15.33:0/4094586038 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368017070 con 0x7f736c070200 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.864+0000 7f735e7fc640 1 -- 172.21.15.33:0/4094586038 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7368005ce0 con 0x7f736c070200 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.864+0000 7f73741fd640 1 -- 172.21.15.33:0/4094586038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f736c1e5380 con 0x7f736c070200 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.864+0000 7f735e7fc640 1 -- 172.21.15.33:0/4094586038 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368007270 con 0x7f736c070200 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.864+0000 7f73741fd640 1 -- 172.21.15.33:0/4094586038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f736c1e5940 con 0x7f736c070200 2024-09-12T01:32:49.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.864+0000 7f735e7fc640 1 -- 172.21.15.33:0/4094586038 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f736801a070 con 0x7f736c070200 2024-09-12T01:32:49.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.864+0000 7f735e7fc640 1 -- 172.21.15.33:0/4094586038 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f736801cc10 con 0x7f736c070200 2024-09-12T01:32:49.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.865+0000 7f73741fd640 1 -- 172.21.15.33:0/4094586038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f736c072ce0 con 0x7f736c070200 2024-09-12T01:32:49.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.867+0000 7f735e7fc640 1 -- 172.21.15.33:0/4094586038 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7368007470 con 0x7f736c070200 2024-09-12T01:32:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.937+0000 7f73741fd640 1 -- 172.21.15.33:0/4094586038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b4779b5c-91a9-4a60-a588-39dad1b2b355", "id": 5} v 0) -- 0x7f736c073a10 con 0x7f736c070200 2024-09-12T01:32:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.941+0000 7f735e7fc640 1 -- 172.21.15.33:0/4094586038 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b4779b5c-91a9-4a60-a588-39dad1b2b355", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f736801c440 con 0x7f736c070200 2024-09-12T01:32:49.940 INFO:teuthology.orchestra.run.smithi033.stdout:5 2024-09-12T01:32:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.942+0000 7f73741fd640 1 -- 172.21.15.33:0/4094586038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c070200 msgr2=0x7f736c0cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.942+0000 7f73741fd640 1 --2- 172.21.15.33:0/4094586038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c070200 0x7f736c0cfd10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f736800c4b0 tx=0x7f736800c970 comp rx=0 tx=0).stop 2024-09-12T01:32:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.942+0000 7f73741fd640 1 -- 172.21.15.33:0/4094586038 shutdown_connections 2024-09-12T01:32:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.942+0000 7f73741fd640 1 --2- 172.21.15.33:0/4094586038 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f736c15b0b0 0x7f736c152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.942+0000 7f73741fd640 1 --2- 172.21.15.33:0/4094586038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f736c073c30 0x7f736c1e4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.942+0000 7f73741fd640 1 --2- 172.21.15.33:0/4094586038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c070200 0x7f736c0cfd10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.942+0000 7f73741fd640 1 -- 172.21.15.33:0/4094586038 >> 172.21.15.33:0/4094586038 conn(0x7f736c0b2c50 msgr2=0x7f736c15d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.942+0000 7f73741fd640 1 -- 172.21.15.33:0/4094586038 shutdown_connections 2024-09-12T01:32:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:49.942+0000 7f73741fd640 1 -- 172.21.15.33:0/4094586038 wait complete. 2024-09-12T01:32:49.952 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new 49bb9733-38d5-4bd9-bbd0-f4a00d34d07e 6 2024-09-12T01:32:50.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.054+0000 7f43e1d6d640 1 Processor -- start 2024-09-12T01:32:50.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.054+0000 7f43e1d6d640 1 -- start start 2024-09-12T01:32:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43e1d6d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43dc15c490 0x7f43dc070580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43e1d6d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43dc0d0a80 0x7f43dc073070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43e1d6d640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43dc0cfbb0 con 0x7f43dc072c10 2024-09-12T01:32:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43e1d6d640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f43dc0735b0 con 0x7f43dc0d0a80 2024-09-12T01:32:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43e1d6d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f43dc06ebb0 con 0x7f43dc15c490 2024-09-12T01:32:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43dc15c490 0x7f43dc070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43daffd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f43dc072c10 0x7f43dc0d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37364/0 (socket says 172.21.15.33:37364) 2024-09-12T01:32:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43daffd640 1 -- 172.21.15.33:0/1797055451 learned_addr learned my addr 172.21.15.33:0/1797055451 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-09-12T01:32:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43dbfff640 1 --2- 172.21.15.33:0/1797055451 >> v2:172.21.15.33:3300/0 conn(0x7f43dc0d0a80 0x7f43dc073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43db7fe640 1 -- 172.21.15.33:0/1797055451 >> v2:172.21.15.33:3300/0 conn(0x7f43dc0d0a80 msgr2=0x7f43dc073070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43db7fe640 1 --2- 172.21.15.33:0/1797055451 >> v2:172.21.15.33:3300/0 conn(0x7f43dc0d0a80 0x7f43dc073070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43db7fe640 1 -- 172.21.15.33:0/1797055451 >> v1:172.21.15.33:6789/0 conn(0x7f43dc072c10 legacy=0x7f43dc0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43da7fc640 1 -- 172.21.15.33:0/1797055451 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3910094994 0 0) 0x7f43dc0cfbb0 con 0x7f43dc072c10 2024-09-12T01:32:50.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43db7fe640 1 -- 172.21.15.33:0/1797055451 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc070ac0 con 0x7f43dc15c490 2024-09-12T01:32:50.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.055+0000 7f43dbfff640 1 --2- 172.21.15.33:0/1797055451 >> v2:172.21.15.33:3300/0 conn(0x7f43dc0d0a80 0x7f43dc073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.056+0000 7f43db7fe640 1 --2- 172.21.15.33:0/1797055451 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43dc15c490 0x7f43dc070580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0097a0 tx=0x7f43cc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac6cc67701030174 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.056+0000 7f43da7fc640 1 -- 172.21.15.33:0/1797055451 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc037070 con 0x7f43dc15c490 2024-09-12T01:32:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.056+0000 7f43da7fc640 1 -- 172.21.15.33:0/1797055451 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc02ae50 con 0x7f43dc15c490 2024-09-12T01:32:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.056+0000 7f43da7fc640 1 -- 172.21.15.33:0/1797055451 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc032660 con 0x7f43dc15c490 2024-09-12T01:32:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.057+0000 7f43e1d6d640 1 -- 172.21.15.33:0/1797055451 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43dc15c490 msgr2=0x7f43dc070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.057+0000 7f43e1d6d640 1 --2- 172.21.15.33:0/1797055451 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43dc15c490 0x7f43dc070580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0097a0 tx=0x7f43cc02a100 comp rx=0 tx=0).stop 2024-09-12T01:32:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.057+0000 7f43e1d6d640 1 -- 172.21.15.33:0/1797055451 shutdown_connections 2024-09-12T01:32:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.057+0000 7f43e1d6d640 1 --2- 172.21.15.33:0/1797055451 >> v2:172.21.15.33:3300/0 conn(0x7f43dc0d0a80 0x7f43dc073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.057+0000 7f43e1d6d640 1 --2- 172.21.15.33:0/1797055451 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43dc15c490 0x7f43dc070580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.057+0000 7f43e1d6d640 1 -- 172.21.15.33:0/1797055451 >> 172.21.15.33:0/1797055451 conn(0x7f43dc0d1100 msgr2=0x7f43dc0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.057+0000 7f43e1d6d640 1 -- 172.21.15.33:0/1797055451 shutdown_connections 2024-09-12T01:32:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.057+0000 7f43e1d6d640 1 -- 172.21.15.33:0/1797055451 wait complete. 2024-09-12T01:32:50.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.058+0000 7f43e1d6d640 1 Processor -- start 2024-09-12T01:32:50.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.058+0000 7f43e1d6d640 1 -- start start 2024-09-12T01:32:50.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.059+0000 7f43e1d6d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43dc072c10 0x7f43dc0c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.059+0000 7f43e1d6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43dc0d0a80 0x7f43dc152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.059+0000 7f43e1d6d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43dc15c490 0x7f43dc0c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.059+0000 7f43e1d6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc0cfe50 con 0x7f43dc0d0a80 2024-09-12T01:32:50.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.059+0000 7f43e1d6d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc06ee00 con 0x7f43dc072c10 2024-09-12T01:32:50.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.059+0000 7f43e1d6d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f43dc0adf30 con 0x7f43dc15c490 2024-09-12T01:32:50.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.059+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43dc072c10 0x7f43dc0c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.059+0000 7f43dbfff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43dc15c490 0x7f43dc0c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.059+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43dc072c10 0x7f43dc0c5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38924/0 (socket says 172.21.15.33:38924) 2024-09-12T01:32:50.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.059+0000 7f43db7fe640 1 -- 172.21.15.33:0/2573366781 learned_addr learned my addr 172.21.15.33:0/2573366781 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:32:50.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.059+0000 7f43dbfff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43dc15c490 0x7f43dc0c51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.33:52994/0 (socket says 172.21.15.33:52994) 2024-09-12T01:32:50.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.059+0000 7f43daffd640 1 --2- 172.21.15.33:0/2573366781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43dc0d0a80 0x7f43dc152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43db7fe640 1 -- 172.21.15.33:0/2573366781 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43dc15c490 msgr2=0x7f43dc0c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43db7fe640 1 --2- 172.21.15.33:0/2573366781 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43dc15c490 0x7f43dc0c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43db7fe640 1 -- 172.21.15.33:0/2573366781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43dc0d0a80 msgr2=0x7f43dc152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43db7fe640 1 --2- 172.21.15.33:0/2573366781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43dc0d0a80 0x7f43dc152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43db7fe640 1 -- 172.21.15.33:0/2573366781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc0c6390 con 0x7f43dc072c10 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43dbfff640 1 --2- 172.21.15.33:0/2573366781 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43dc15c490 0x7f43dc0c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43db7fe640 1 --2- 172.21.15.33:0/2573366781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43dc072c10 0x7f43dc0c5e50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0098c0 tx=0x7f43cc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43c3fff640 1 -- 172.21.15.33:0/2573366781 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc042070 con 0x7f43dc072c10 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43c3fff640 1 -- 172.21.15.33:0/2573366781 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc03d440 con 0x7f43dc072c10 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43e1d6d640 1 -- 172.21.15.33:0/2573366781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43dc09b800 con 0x7f43dc072c10 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43c3fff640 1 -- 172.21.15.33:0/2573366781 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc037040 con 0x7f43dc072c10 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43e1d6d640 1 -- 172.21.15.33:0/2573366781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43dc09d210 con 0x7f43dc072c10 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43c3fff640 1 -- 172.21.15.33:0/2573366781 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f43cc045070 con 0x7f43dc072c10 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.060+0000 7f43c3fff640 1 -- 172.21.15.33:0/2573366781 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f43cc047c70 con 0x7f43dc072c10 2024-09-12T01:32:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.061+0000 7f43e1d6d640 1 -- 172.21.15.33:0/2573366781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43dc073070 con 0x7f43dc072c10 2024-09-12T01:32:50.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.065+0000 7f43c3fff640 1 -- 172.21.15.33:0/2573366781 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f43cc047890 con 0x7f43dc072c10 2024-09-12T01:32:50.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.135+0000 7f43e1d6d640 1 -- 172.21.15.33:0/2573366781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd new", "uuid": "49bb9733-38d5-4bd9-bbd0-f4a00d34d07e", "id": 6} v 0) -- 0x7f43dc0aedb0 con 0x7f43dc072c10 2024-09-12T01:32:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.138+0000 7f43c3fff640 1 -- 172.21.15.33:0/2573366781 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "49bb9733-38d5-4bd9-bbd0-f4a00d34d07e", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f43cc047a70 con 0x7f43dc072c10 2024-09-12T01:32:50.138 INFO:teuthology.orchestra.run.smithi033.stdout:6 2024-09-12T01:32:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.139+0000 7f43e1d6d640 1 -- 172.21.15.33:0/2573366781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43dc072c10 msgr2=0x7f43dc0c5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.139+0000 7f43e1d6d640 1 --2- 172.21.15.33:0/2573366781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43dc072c10 0x7f43dc0c5e50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0098c0 tx=0x7f43cc0046c0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.139+0000 7f43e1d6d640 1 -- 172.21.15.33:0/2573366781 shutdown_connections 2024-09-12T01:32:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.139+0000 7f43e1d6d640 1 --2- 172.21.15.33:0/2573366781 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43dc15c490 0x7f43dc0c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.139+0000 7f43e1d6d640 1 --2- 172.21.15.33:0/2573366781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43dc0d0a80 0x7f43dc152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.139+0000 7f43e1d6d640 1 --2- 172.21.15.33:0/2573366781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43dc072c10 0x7f43dc0c5e50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.139+0000 7f43e1d6d640 1 -- 172.21.15.33:0/2573366781 >> 172.21.15.33:0/2573366781 conn(0x7f43dc0d1100 msgr2=0x7f43dc0b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.139+0000 7f43e1d6d640 1 -- 172.21.15.33:0/2573366781 shutdown_connections 2024-09-12T01:32:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.139+0000 7f43e1d6d640 1 -- 172.21.15.33:0/2573366781 wait complete. 2024-09-12T01:32:50.149 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new 1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d 7 2024-09-12T01:32:50.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.252+0000 7fbbd9cba640 1 Processor -- start 2024-09-12T01:32:50.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.252+0000 7fbbd9cba640 1 -- start start 2024-09-12T01:32:50.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.253+0000 7fbbd9cba640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbbd40af880 0x7fbbd40afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.253+0000 7fbbd9cba640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbbd415e450 0x7fbbd40b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.253+0000 7fbbd9cba640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbd40b01a0 con 0x7fbbd40b5f90 2024-09-12T01:32:50.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.253+0000 7fbbd9cba640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fbbd40b64a0 con 0x7fbbd40af880 2024-09-12T01:32:50.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.253+0000 7fbbd9cba640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd40b6780 con 0x7fbbd415e450 2024-09-12T01:32:50.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.253+0000 7fbbd37fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbbd40af880 0x7fbbd40afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.253+0000 7fbbd2ffd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbbd415e450 0x7fbbd40b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.253+0000 7fbbd3fff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbbd40b5f90 0x7fbbd40ad8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37376/0 (socket says 172.21.15.33:37376) 2024-09-12T01:32:50.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.253+0000 7fbbd3fff640 1 -- 172.21.15.33:0/359468937 learned_addr learned my addr 172.21.15.33:0/359468937 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-09-12T01:32:50.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.253+0000 7fbbd37fe640 1 -- 172.21.15.33:0/359468937 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbbd415e450 msgr2=0x7fbbd40b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:50.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.253+0000 7fbbd37fe640 1 --2- 172.21.15.33:0/359468937 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbbd415e450 0x7fbbd40b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.253+0000 7fbbd37fe640 1 -- 172.21.15.33:0/359468937 >> v1:172.21.15.33:6789/0 conn(0x7fbbd40b5f90 legacy=0x7fbbd40ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.254+0000 7fbbd27fc640 1 -- 172.21.15.33:0/359468937 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1946893223 0 0) 0x7fbbd40b01a0 con 0x7fbbd40b5f90 2024-09-12T01:32:50.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.254+0000 7fbbd37fe640 1 -- 172.21.15.33:0/359468937 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd40b6a60 con 0x7fbbd40af880 2024-09-12T01:32:50.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.254+0000 7fbbd37fe640 1 --2- 172.21.15.33:0/359468937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd40af880 0x7fbbd40afc60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40097a0 tx=0x7fbbc402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92c552c9e669ae7c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:50.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.254+0000 7fbbd27fc640 1 -- 172.21.15.33:0/359468937 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4037070 con 0x7fbbd40af880 2024-09-12T01:32:50.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.254+0000 7fbbd27fc640 1 -- 172.21.15.33:0/359468937 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc4032440 con 0x7fbbd40af880 2024-09-12T01:32:50.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.254+0000 7fbbd27fc640 1 -- 172.21.15.33:0/359468937 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4032810 con 0x7fbbd40af880 2024-09-12T01:32:50.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.255+0000 7fbbd9cba640 1 -- 172.21.15.33:0/359468937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd40af880 msgr2=0x7fbbd40afc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.255+0000 7fbbd9cba640 1 --2- 172.21.15.33:0/359468937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd40af880 0x7fbbd40afc60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40097a0 tx=0x7fbbc402a180 comp rx=0 tx=0).stop 2024-09-12T01:32:50.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.255+0000 7fbbd9cba640 1 -- 172.21.15.33:0/359468937 shutdown_connections 2024-09-12T01:32:50.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.255+0000 7fbbd9cba640 1 --2- 172.21.15.33:0/359468937 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbbd415e450 0x7fbbd40b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.255+0000 7fbbd9cba640 1 --2- 172.21.15.33:0/359468937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd40af880 0x7fbbd40afc60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.255+0000 7fbbd9cba640 1 -- 172.21.15.33:0/359468937 >> 172.21.15.33:0/359468937 conn(0x7fbbd4073370 msgr2=0x7fbbd406baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:50.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.255+0000 7fbbd9cba640 1 -- 172.21.15.33:0/359468937 shutdown_connections 2024-09-12T01:32:50.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.255+0000 7fbbd9cba640 1 -- 172.21.15.33:0/359468937 wait complete. 2024-09-12T01:32:50.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.256+0000 7fbbd9cba640 1 Processor -- start 2024-09-12T01:32:50.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.257+0000 7fbbd9cba640 1 -- start start 2024-09-12T01:32:50.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.257+0000 7fbbd9cba640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbbd40af880 0x7fbbd415ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.257+0000 7fbbd9cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbd40b5f90 0x7fbbd41531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.257+0000 7fbbd9cba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd415e450 0x7fbbd41536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.257+0000 7fbbd9cba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd4070770 con 0x7fbbd40b5f90 2024-09-12T01:32:50.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.257+0000 7fbbd9cba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd406e8f0 con 0x7fbbd415e450 2024-09-12T01:32:50.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.257+0000 7fbbd9cba640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd40708f0 con 0x7fbbd40af880 2024-09-12T01:32:50.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.257+0000 7fbbd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd415e450 0x7fbbd41536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.257+0000 7fbbd37fe640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbbd40af880 0x7fbbd415ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.257+0000 7fbbd37fe640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbbd40af880 0x7fbbd415ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.33:53008/0 (socket says 172.21.15.33:53008) 2024-09-12T01:32:50.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.257+0000 7fbbd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd415e450 0x7fbbd41536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38940/0 (socket says 172.21.15.33:38940) 2024-09-12T01:32:50.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.257+0000 7fbbd37fe640 1 -- 172.21.15.33:0/3663765941 learned_addr learned my addr 172.21.15.33:0/3663765941 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-12T01:32:50.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbd2ffd640 1 --2- 172.21.15.33:0/3663765941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbd40b5f90 0x7fbbd41531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbd37fe640 1 -- 172.21.15.33:0/3663765941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd415e450 msgr2=0x7fbbd41536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbd37fe640 1 --2- 172.21.15.33:0/3663765941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd415e450 0x7fbbd41536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbd37fe640 1 -- 172.21.15.33:0/3663765941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbd40b5f90 msgr2=0x7fbbd41531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbd37fe640 1 --2- 172.21.15.33:0/3663765941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbd40b5f90 0x7fbbd41531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbd37fe640 1 -- 172.21.15.33:0/3663765941 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd40cee20 con 0x7fbbd40af880 2024-09-12T01:32:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbd3fff640 1 --2- 172.21.15.33:0/3663765941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd415e450 0x7fbbd41536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbd37fe640 1 --2- 172.21.15.33:0/3663765941 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbbd40af880 0x7fbbd415ab30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40098c0 tx=0x7fbbc40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbbbfff640 1 -- 172.21.15.33:0/3663765941 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4037040 con 0x7fbbd40af880 2024-09-12T01:32:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbbbfff640 1 -- 172.21.15.33:0/3663765941 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc4030500 con 0x7fbbd40af880 2024-09-12T01:32:50.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbd9cba640 1 -- 172.21.15.33:0/3663765941 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd41e8d10 con 0x7fbbd40af880 2024-09-12T01:32:50.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbbbfff640 1 -- 172.21.15.33:0/3663765941 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc40308d0 con 0x7fbbd40af880 2024-09-12T01:32:50.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbd9cba640 1 -- 172.21.15.33:0/3663765941 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd40cf380 con 0x7fbbd40af880 2024-09-12T01:32:50.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbbbfff640 1 -- 172.21.15.33:0/3663765941 <== mon.2 v2:172.21.15.33:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbbc4044020 con 0x7fbbd40af880 2024-09-12T01:32:50.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.258+0000 7fbbbbfff640 1 -- 172.21.15.33:0/3663765941 <== mon.2 v2:172.21.15.33:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fbbc403bbb0 con 0x7fbbd40af880 2024-09-12T01:32:50.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.259+0000 7fbbd9cba640 1 -- 172.21.15.33:0/3663765941 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd40b5a50 con 0x7fbbd40af880 2024-09-12T01:32:50.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.263+0000 7fbbbbfff640 1 -- 172.21.15.33:0/3663765941 <== mon.2 v2:172.21.15.33:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbbd40cee20 con 0x7fbbd40af880 2024-09-12T01:32:50.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.334+0000 7fbbd9cba640 1 -- 172.21.15.33:0/3663765941 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d", "id": 7} v 0) -- 0x7fbbd4070c20 con 0x7fbbd40af880 2024-09-12T01:32:50.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.341+0000 7fbbbbfff640 1 -- 172.21.15.33:0/3663765941 <== mon.2 v2:172.21.15.33:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fbbc403b430 con 0x7fbbd40af880 2024-09-12T01:32:50.341 INFO:teuthology.orchestra.run.smithi033.stdout:7 2024-09-12T01:32:50.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.342+0000 7fbbd9cba640 1 -- 172.21.15.33:0/3663765941 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbbd40af880 msgr2=0x7fbbd415ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.342+0000 7fbbd9cba640 1 --2- 172.21.15.33:0/3663765941 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbbd40af880 0x7fbbd415ab30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40098c0 tx=0x7fbbc40046c0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.342+0000 7fbbd9cba640 1 -- 172.21.15.33:0/3663765941 shutdown_connections 2024-09-12T01:32:50.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.342+0000 7fbbd9cba640 1 --2- 172.21.15.33:0/3663765941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd415e450 0x7fbbd41536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.342+0000 7fbbd9cba640 1 --2- 172.21.15.33:0/3663765941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbd40b5f90 0x7fbbd41531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.342+0000 7fbbd9cba640 1 --2- 172.21.15.33:0/3663765941 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbbd40af880 0x7fbbd415ab30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.342+0000 7fbbd9cba640 1 -- 172.21.15.33:0/3663765941 >> 172.21.15.33:0/3663765941 conn(0x7fbbd4073370 msgr2=0x7fbbd406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:50.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.342+0000 7fbbd9cba640 1 -- 172.21.15.33:0/3663765941 shutdown_connections 2024-09-12T01:32:50.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:32:50.342+0000 7fbbd9cba640 1 -- 172.21.15.33:0/3663765941 wait complete. 2024-09-12T01:32:50.352 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-12T01:32:50.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-12T01:32:50.354 INFO:tasks.ceph.osd.0:Started 2024-09-12T01:32:50.354 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-12T01:32:50.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-12T01:32:50.356 INFO:tasks.ceph.osd.1:Started 2024-09-12T01:32:50.357 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-12T01:32:50.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-12T01:32:50.359 INFO:tasks.ceph.osd.2:Started 2024-09-12T01:32:50.360 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-12T01:32:50.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-12T01:32:50.363 INFO:tasks.ceph.osd.3:Started 2024-09-12T01:32:50.363 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-12T01:32:50.363 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-12T01:32:50.366 INFO:tasks.ceph.osd.4:Started 2024-09-12T01:32:50.367 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-12T01:32:50.367 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-12T01:32:50.369 INFO:tasks.ceph.osd.5:Started 2024-09-12T01:32:50.369 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-12T01:32:50.369 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-12T01:32:50.371 INFO:tasks.ceph.osd.6:Started 2024-09-12T01:32:50.372 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-12T01:32:50.372 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-12T01:32:50.374 INFO:tasks.ceph.osd.7:Started 2024-09-12T01:32:50.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:32:50.452 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:50.450+0000 7fa2fe01c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:32:50.507 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:50.508+0000 7fc061c21040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:32:50.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.509+0000 7fd50756d640 1 Processor -- start 2024-09-12T01:32:50.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.509+0000 7fd50756d640 1 -- start start 2024-09-12T01:32:50.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd50756d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5000cdaf0 0x7fd5000cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd50756d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd500073ab0 0x7fd5000d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd505309640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5000cdaf0 0x7fd5000cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd504b08640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd500073620 0x7fd5000cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46034/0 (socket says 172.21.15.16:46034) 2024-09-12T01:32:50.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd504b08640 1 -- 172.21.15.16:0/3980215347 learned_addr learned my addr 172.21.15.16:0/3980215347 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:32:50.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd50756d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5000cf080 con 0x7fd500073620 2024-09-12T01:32:50.510 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fa2fe01c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:32:50.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd50756d640 1 -- 172.21.15.16:0/3980215347 --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd5000ce9e0 con 0x7fd5000cdaf0 2024-09-12T01:32:50.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd50756d640 1 -- 172.21.15.16:0/3980215347 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fd5000ce410 con 0x7fd500073ab0 2024-09-12T01:32:50.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd505b0a640 1 --2- 172.21.15.16:0/3980215347 >> v2:172.21.15.33:3300/0 conn(0x7fd500073ab0 0x7fd5000d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd4f7fff640 1 -- 172.21.15.16:0/3980215347 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1763919264 0 0) 0x7fd5000cf080 con 0x7fd500073620 2024-09-12T01:32:50.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd4f7fff640 1 -- 172.21.15.16:0/3980215347 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4ec003520 con 0x7fd500073620 2024-09-12T01:32:50.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd505309640 1 -- 172.21.15.16:0/3980215347 >> v2:172.21.15.33:3300/0 conn(0x7fd500073ab0 msgr2=0x7fd5000d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:50.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd505309640 1 --2- 172.21.15.16:0/3980215347 >> v2:172.21.15.33:3300/0 conn(0x7fd500073ab0 0x7fd5000d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd505309640 1 -- 172.21.15.16:0/3980215347 >> v1:172.21.15.16:6789/0 conn(0x7fd500073620 legacy=0x7fd5000cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.510+0000 7fd505309640 1 -- 172.21.15.16:0/3980215347 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5000b43a0 con 0x7fd5000cdaf0 2024-09-12T01:32:50.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.511+0000 7fd505309640 1 --2- 172.21.15.16:0/3980215347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5000cdaf0 0x7fd5000cded0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd50006df60 tx=0x7fd4f002c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd865142a46be807 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:50.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.511+0000 7fd4f7fff640 1 -- 172.21.15.16:0/3980215347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0038070 con 0x7fd5000cdaf0 2024-09-12T01:32:50.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.511+0000 7fd4f7fff640 1 -- 172.21.15.16:0/3980215347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f0031710 con 0x7fd5000cdaf0 2024-09-12T01:32:50.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.511+0000 7fd4f7fff640 1 -- 172.21.15.16:0/3980215347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0031ae0 con 0x7fd5000cdaf0 2024-09-12T01:32:50.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.512+0000 7fd50756d640 1 -- 172.21.15.16:0/3980215347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5000cdaf0 msgr2=0x7fd5000cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.512+0000 7fd50756d640 1 --2- 172.21.15.16:0/3980215347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5000cdaf0 0x7fd5000cded0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd50006df60 tx=0x7fd4f002c5c0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.512+0000 7fd50756d640 1 -- 172.21.15.16:0/3980215347 shutdown_connections 2024-09-12T01:32:50.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.512+0000 7fd50756d640 1 --2- 172.21.15.16:0/3980215347 >> v2:172.21.15.33:3300/0 conn(0x7fd500073ab0 0x7fd5000d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.512+0000 7fd50756d640 1 --2- 172.21.15.16:0/3980215347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5000cdaf0 0x7fd5000cded0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.512+0000 7fd50756d640 1 -- 172.21.15.16:0/3980215347 >> 172.21.15.16:0/3980215347 conn(0x7fd5000b2c50 msgr2=0x7fd5000b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:50.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.512+0000 7fd50756d640 1 -- 172.21.15.16:0/3980215347 shutdown_connections 2024-09-12T01:32:50.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.512+0000 7fd50756d640 1 -- 172.21.15.16:0/3980215347 wait complete. 2024-09-12T01:32:50.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.513+0000 7fd50756d640 1 Processor -- start 2024-09-12T01:32:50.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.513+0000 7fd50756d640 1 -- start start 2024-09-12T01:32:50.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd50756d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd500073620 0x7fd50013c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd50756d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd500073ab0 0x7fd5000cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd50756d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5000cdaf0 0x7fd50015d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd50756d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5000b4800 con 0x7fd5000cdaf0 2024-09-12T01:32:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd50756d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd5000b4680 con 0x7fd500073ab0 2024-09-12T01:32:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd50756d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fd500144de0 con 0x7fd500073620 2024-09-12T01:32:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd504b08640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd500073ab0 0x7fd5000cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd505b0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5000cdaf0 0x7fd50015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd504b08640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd500073ab0 0x7fd5000cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:50768/0 (socket says 172.21.15.16:50768) 2024-09-12T01:32:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd504b08640 1 -- 172.21.15.16:0/3501919626 learned_addr learned my addr 172.21.15.16:0/3501919626 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:32:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd505309640 1 --2- 172.21.15.16:0/3501919626 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd500073620 0x7fd50013c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd504b08640 1 -- 172.21.15.16:0/3501919626 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd500073620 msgr2=0x7fd50013c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd504b08640 1 --2- 172.21.15.16:0/3501919626 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd500073620 0x7fd50013c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd504b08640 1 -- 172.21.15.16:0/3501919626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5000cdaf0 msgr2=0x7fd50015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.514+0000 7fd504b08640 1 --2- 172.21.15.16:0/3501919626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5000cdaf0 0x7fd50015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.515+0000 7fd504b08640 1 -- 172.21.15.16:0/3501919626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50013c540 con 0x7fd500073ab0 2024-09-12T01:32:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.515+0000 7fd505b0a640 1 --2- 172.21.15.16:0/3501919626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5000cdaf0 0x7fd50015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.515+0000 7fd504b08640 1 --2- 172.21.15.16:0/3501919626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd500073ab0 0x7fd5000cb5e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00c400 tx=0x7fd4fc00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.515+0000 7fd4f5ffb640 1 -- 172.21.15.16:0/3501919626 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc017070 con 0x7fd500073ab0 2024-09-12T01:32:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.515+0000 7fd4f5ffb640 1 -- 172.21.15.16:0/3501919626 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4fc005ce0 con 0x7fd500073ab0 2024-09-12T01:32:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.515+0000 7fd4f5ffb640 1 -- 172.21.15.16:0/3501919626 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc0072d0 con 0x7fd500073ab0 2024-09-12T01:32:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.515+0000 7fd50756d640 1 -- 172.21.15.16:0/3501919626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50015c110 con 0x7fd500073ab0 2024-09-12T01:32:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.515+0000 7fd4f5ffb640 1 -- 172.21.15.16:0/3501919626 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4fc01a070 con 0x7fd500073ab0 2024-09-12T01:32:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.515+0000 7fd50756d640 1 -- 172.21.15.16:0/3501919626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd50015c6d0 con 0x7fd500073ab0 2024-09-12T01:32:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.515+0000 7fd4f5ffb640 1 -- 172.21.15.16:0/3501919626 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd4fc012d80 con 0x7fd500073ab0 2024-09-12T01:32:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.516+0000 7fd50756d640 1 -- 172.21.15.16:0/3501919626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5000b3f50 con 0x7fd500073ab0 2024-09-12T01:32:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.518+0000 7fd4f5ffb640 1 -- 172.21.15.16:0/3501919626 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd4fc022020 con 0x7fd500073ab0 2024-09-12T01:32:50.563 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:50.563+0000 7fc061c21040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:32:50.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.579+0000 7fd50756d640 1 -- 172.21.15.16:0/3501919626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd50015ea10 con 0x7fd500073ab0 2024-09-12T01:32:50.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.579+0000 7fd4f5ffb640 1 -- 172.21.15.16:0/3501919626 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd4fc0074c0 con 0x7fd500073ab0 2024-09-12T01:32:50.579 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:32:50.579 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:32:50.336270+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:32:50.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.580+0000 7fd50756d640 1 -- 172.21.15.16:0/3501919626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd500073ab0 msgr2=0x7fd5000cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.580+0000 7fd50756d640 1 --2- 172.21.15.16:0/3501919626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd500073ab0 0x7fd5000cb5e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00c400 tx=0x7fd4fc00c8c0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.580+0000 7fd50756d640 1 -- 172.21.15.16:0/3501919626 shutdown_connections 2024-09-12T01:32:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.580+0000 7fd50756d640 1 --2- 172.21.15.16:0/3501919626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5000cdaf0 0x7fd50015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.580+0000 7fd50756d640 1 --2- 172.21.15.16:0/3501919626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd500073ab0 0x7fd5000cb5e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.580+0000 7fd50756d640 1 --2- 172.21.15.16:0/3501919626 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd500073620 0x7fd50013c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.580+0000 7fd50756d640 1 -- 172.21.15.16:0/3501919626 >> 172.21.15.16:0/3501919626 conn(0x7fd5000b2c50 msgr2=0x7fd5000cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.580+0000 7fd50756d640 1 -- 172.21.15.16:0/3501919626 shutdown_connections 2024-09-12T01:32:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:32:50.580+0000 7fd50756d640 1 -- 172.21.15.16:0/3501919626 wait complete. 2024-09-12T01:32:50.594 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-12T01:32:50.594 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-12T01:32:50.898 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:50.898+0000 7fa2fe01c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:32:50.946 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:50.946+0000 7fc061c21040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:32:50.986 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:50.986+0000 7fa2fe01c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:32:51.034 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:51.035+0000 7fc061c21040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:32:51.099 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:51.100+0000 7fa2fe01c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:32:51.145 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:51.146+0000 7fc061c21040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:32:51.148 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:51.148+0000 7fa2fe01c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:32:51.193 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:51.194+0000 7fc061c21040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:32:51.267 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:51.267+0000 7fa2fe01c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:32:51.312 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:51.313+0000 7fc061c21040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:32:51.323 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:51.323+0000 7fa2fe01c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:32:51.367 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:51.368+0000 7fc061c21040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:32:51.370 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:51.370+0000 7fa2fe01c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:32:51.400 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T01:32:51.414 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:51.415+0000 7fc061c21040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:32:51.489 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:51.490+0000 7fa2fe01c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:32:51.494 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.494+0000 7fb5199e8640 1 Processor -- start 2024-09-12T01:32:51.494 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.494+0000 7fb5199e8640 1 -- start start 2024-09-12T01:32:51.495 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb5199e8640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5141628e0 0x7fb5140d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:51.495 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb5199e8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5140d45d0 0x7fb5140d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:51.495 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb5199e8640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5140d3060 con 0x7fb514162500 2024-09-12T01:32:51.495 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb5199e8640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb51415f700 con 0x7fb5141628e0 2024-09-12T01:32:51.495 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb5199e8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb514162d50 con 0x7fb5140d45d0 2024-09-12T01:32:51.495 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb5127fc640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5141628e0 0x7fb5140d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:51.495 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb5137fe640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5140d45d0 0x7fb5140d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:51.496 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb512ffd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb514162500 0x7fb5140cfbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:47332/0 (socket says 172.21.15.16:47332) 2024-09-12T01:32:51.496 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb512ffd640 1 -- 172.21.15.16:0/1597688212 learned_addr learned my addr 172.21.15.16:0/1597688212 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:32:51.496 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb5127fc640 1 -- 172.21.15.16:0/1597688212 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5140d45d0 msgr2=0x7fb5140d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:51.496 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb5127fc640 1 --2- 172.21.15.16:0/1597688212 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5140d45d0 0x7fb5140d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:51.496 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb5127fc640 1 -- 172.21.15.16:0/1597688212 >> v1:172.21.15.33:6789/0 conn(0x7fb514162500 legacy=0x7fb5140cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:51.496 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb5127fc640 1 -- 172.21.15.16:0/1597688212 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5140cf100 con 0x7fb5141628e0 2024-09-12T01:32:51.496 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.495+0000 7fb511ffb640 1 -- 172.21.15.16:0/1597688212 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 273406158 0 0) 0x7fb5140d3060 con 0x7fb514162500 2024-09-12T01:32:51.497 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.496+0000 7fb5127fc640 1 --2- 172.21.15.16:0/1597688212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5141628e0 0x7fb5140d4090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb5080097d0 tx=0x7fb50802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=640ed48b830e81b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:51.497 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.496+0000 7fb511ffb640 1 -- 172.21.15.16:0/1597688212 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508037070 con 0x7fb5141628e0 2024-09-12T01:32:51.497 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.496+0000 7fb511ffb640 1 -- 172.21.15.16:0/1597688212 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50802ae40 con 0x7fb5141628e0 2024-09-12T01:32:51.497 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.496+0000 7fb511ffb640 1 -- 172.21.15.16:0/1597688212 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508032660 con 0x7fb5141628e0 2024-09-12T01:32:51.497 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.497+0000 7fb5199e8640 1 -- 172.21.15.16:0/1597688212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5141628e0 msgr2=0x7fb5140d4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:51.497 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.497+0000 7fb5199e8640 1 --2- 172.21.15.16:0/1597688212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5141628e0 0x7fb5140d4090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb5080097d0 tx=0x7fb50802a130 comp rx=0 tx=0).stop 2024-09-12T01:32:51.498 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.497+0000 7fb5199e8640 1 -- 172.21.15.16:0/1597688212 shutdown_connections 2024-09-12T01:32:51.498 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.497+0000 7fb5199e8640 1 --2- 172.21.15.16:0/1597688212 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5140d45d0 0x7fb5140d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:51.498 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.497+0000 7fb5199e8640 1 --2- 172.21.15.16:0/1597688212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5141628e0 0x7fb5140d4090 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:51.498 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.497+0000 7fb5199e8640 1 -- 172.21.15.16:0/1597688212 >> 172.21.15.16:0/1597688212 conn(0x7fb51406c3f0 msgr2=0x7fb51409d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:51.498 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.497+0000 7fb5199e8640 1 -- 172.21.15.16:0/1597688212 shutdown_connections 2024-09-12T01:32:51.498 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.497+0000 7fb5199e8640 1 -- 172.21.15.16:0/1597688212 wait complete. 2024-09-12T01:32:51.500 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.498+0000 7fb5199e8640 1 Processor -- start 2024-09-12T01:32:51.500 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.498+0000 7fb5199e8640 1 -- start start 2024-09-12T01:32:51.500 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.499+0000 7fb5199e8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5140d45d0 0x7fb5140c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:51.500 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.499+0000 7fb5199e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514162500 0x7fb5141f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:51.501 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.499+0000 7fb5199e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5141628e0 0x7fb5140c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:51.501 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.499+0000 7fb5199e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5140d4a00 con 0x7fb514162500 2024-09-12T01:32:51.501 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.499+0000 7fb5199e8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5140cf6c0 con 0x7fb5141628e0 2024-09-12T01:32:51.501 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.499+0000 7fb5199e8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb5140d3340 con 0x7fb5140d45d0 2024-09-12T01:32:51.501 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.499+0000 7fb5127fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514162500 0x7fb5141f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:51.501 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.499+0000 7fb5127fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514162500 0x7fb5141f4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55088/0 (socket says 172.21.15.16:55088) 2024-09-12T01:32:51.501 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.499+0000 7fb5127fc640 1 -- 172.21.15.16:0/3472463968 learned_addr learned my addr 172.21.15.16:0/3472463968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:32:51.501 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.499+0000 7fb5137fe640 1 --2- 172.21.15.16:0/3472463968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5141628e0 0x7fb5140c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:51.501 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.499+0000 7fb512ffd640 1 --2- 172.21.15.16:0/3472463968 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5140d45d0 0x7fb5140c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:51.501 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb5137fe640 1 -- 172.21.15.16:0/3472463968 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5140d45d0 msgr2=0x7fb5140c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:51.501 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb5137fe640 1 --2- 172.21.15.16:0/3472463968 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5140d45d0 0x7fb5140c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:51.501 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb5137fe640 1 -- 172.21.15.16:0/3472463968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514162500 msgr2=0x7fb5141f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:51.501 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb5137fe640 1 --2- 172.21.15.16:0/3472463968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514162500 0x7fb5141f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:51.502 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb5137fe640 1 -- 172.21.15.16:0/3472463968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5140b88f0 con 0x7fb5141628e0 2024-09-12T01:32:51.502 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb5127fc640 1 --2- 172.21.15.16:0/3472463968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514162500 0x7fb5141f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:51.502 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb5137fe640 1 --2- 172.21.15.16:0/3472463968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5141628e0 0x7fb5140c9cb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00efc0 tx=0x7fb50c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:51.502 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb4fbfff640 1 -- 172.21.15.16:0/3472463968 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c018020 con 0x7fb5141628e0 2024-09-12T01:32:51.502 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb4fbfff640 1 -- 172.21.15.16:0/3472463968 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50c014480 con 0x7fb5141628e0 2024-09-12T01:32:51.502 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb4fbfff640 1 -- 172.21.15.16:0/3472463968 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c014870 con 0x7fb5141628e0 2024-09-12T01:32:51.502 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb5199e8640 1 -- 172.21.15.16:0/3472463968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5140b8bd0 con 0x7fb5141628e0 2024-09-12T01:32:51.502 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb5199e8640 1 -- 172.21.15.16:0/3472463968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5140b9190 con 0x7fb5141628e0 2024-09-12T01:32:51.502 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb4fbfff640 1 -- 172.21.15.16:0/3472463968 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb50c005d10 con 0x7fb5141628e0 2024-09-12T01:32:51.502 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.500+0000 7fb4fbfff640 1 -- 172.21.15.16:0/3472463968 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fb50c013ab0 con 0x7fb5141628e0 2024-09-12T01:32:51.502 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.501+0000 7fb5199e8640 1 -- 172.21.15.16:0/3472463968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5140d4090 con 0x7fb5141628e0 2024-09-12T01:32:51.503 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.503+0000 7fb4fbfff640 1 -- 172.21.15.16:0/3472463968 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb50c022030 con 0x7fb5141628e0 2024-09-12T01:32:51.531 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:51.532+0000 7fc061c21040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:32:51.543 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:51.543+0000 7fa2fe01c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:32:51.563 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.563+0000 7fb5199e8640 1 -- 172.21.15.16:0/3472463968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb51406ce30 con 0x7fb5141628e0 2024-09-12T01:32:51.563 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.564+0000 7fb4fbfff640 1 -- 172.21.15.16:0/3472463968 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fb50c010040 con 0x7fb5141628e0 2024-09-12T01:32:51.563 INFO:teuthology.misc.health.smithi016.stdout: 2024-09-12T01:32:51.564 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":10,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:32:50.336270+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:32:51.564 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.565+0000 7fb5199e8640 1 -- 172.21.15.16:0/3472463968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5141628e0 msgr2=0x7fb5140c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:51.565 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.565+0000 7fb5199e8640 1 --2- 172.21.15.16:0/3472463968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5141628e0 0x7fb5140c9cb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00efc0 tx=0x7fb50c00c490 comp rx=0 tx=0).stop 2024-09-12T01:32:51.565 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.565+0000 7fb5199e8640 1 -- 172.21.15.16:0/3472463968 shutdown_connections 2024-09-12T01:32:51.565 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.565+0000 7fb5199e8640 1 --2- 172.21.15.16:0/3472463968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5141628e0 0x7fb5140c9cb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:51.565 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.565+0000 7fb5199e8640 1 --2- 172.21.15.16:0/3472463968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514162500 0x7fb5141f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:51.565 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.565+0000 7fb5199e8640 1 --2- 172.21.15.16:0/3472463968 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5140d45d0 0x7fb5140c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:51.565 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.565+0000 7fb5199e8640 1 -- 172.21.15.16:0/3472463968 >> 172.21.15.16:0/3472463968 conn(0x7fb51406c3f0 msgr2=0x7fb5140d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:51.566 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.565+0000 7fb5199e8640 1 -- 172.21.15.16:0/3472463968 shutdown_connections 2024-09-12T01:32:51.566 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:51.565+0000 7fb5199e8640 1 -- 172.21.15.16:0/3472463968 wait complete. 2024-09-12T01:32:51.573 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-12T01:32:51.575 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-12T01:32:51.576+0000 7f19f82ef640 -1 Falling back to public interface 2024-09-12T01:32:51.579 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-12T01:32:51.579+0000 7f42369d9640 -1 Falling back to public interface 2024-09-12T01:32:51.582 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-12T01:32:51.583+0000 7f702a624640 -1 Falling back to public interface 2024-09-12T01:32:51.584 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:51.584+0000 7fc061c21040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:32:51.604 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:51.604+0000 7fa2fe01c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:32:51.637 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:51.638+0000 7fc061c21040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:32:51.675 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:51.675+0000 7fa2fe01c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:32:51.707 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:51.708+0000 7fc061c21040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:32:51.721 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:51.722+0000 7fa2fe01c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:32:51.754 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:51.754+0000 7fc061c21040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:32:51.767 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:51.768+0000 7fa2fe01c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:32:51.799 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:51.800+0000 7fc061c21040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:32:51.816 INFO:tasks.ceph.osd.7.smithi033.stderr:2024-09-12T01:32:51.816+0000 7f83b1044640 -1 Falling back to public interface 2024-09-12T01:32:51.824 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-12T01:32:51.824+0000 7f1ebb934640 -1 Falling back to public interface 2024-09-12T01:32:51.827 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-12T01:32:51.827+0000 7f1a6fe1e640 -1 Falling back to public interface 2024-09-12T01:32:51.848 INFO:tasks.ceph.osd.6.smithi033.stderr:2024-09-12T01:32:51.848+0000 7f7ea89fd640 -1 Falling back to public interface 2024-09-12T01:32:51.848 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-12T01:32:51.849+0000 7f91777d9640 -1 Falling back to public interface 2024-09-12T01:32:51.850 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-12T01:32:51.850+0000 7f9176781640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6822/0: (98) Address already in use 2024-09-12T01:32:52.459 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:52.460+0000 7fa2fe01c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:32:52.487 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-12T01:32:52.488+0000 7fc061c21040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:32:53.473 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:53.474+0000 7fa29dd09640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-12T01:32:53.474 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-12T01:32:53.474+0000 7fa29dd09640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-12T01:32:55.346 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-12T01:32:55.347+0000 7f1a6fe1e640 -1 osd.2 0 log_to_monitors true 2024-09-12T01:32:55.394 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-12T01:32:55.394+0000 7f19f82ef640 -1 osd.0 0 log_to_monitors true 2024-09-12T01:32:55.631 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-12T01:32:55.632+0000 7f1ebb934640 -1 osd.5 0 log_to_monitors true 2024-09-12T01:32:55.672 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-12T01:32:55.672+0000 7f42369d9640 -1 osd.3 0 log_to_monitors true 2024-09-12T01:32:55.678 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-12T01:32:55.678+0000 7f91777d9640 -1 osd.4 0 log_to_monitors true 2024-09-12T01:32:55.889 INFO:tasks.ceph.osd.7.smithi033.stderr:2024-09-12T01:32:55.889+0000 7f83b1044640 -1 osd.7 0 log_to_monitors true 2024-09-12T01:32:55.932 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-12T01:32:55.932+0000 7f702a624640 -1 osd.1 0 log_to_monitors true 2024-09-12T01:32:56.178 INFO:tasks.ceph.osd.6.smithi033.stderr:2024-09-12T01:32:56.178+0000 7f7ea89fd640 -1 osd.6 0 log_to_monitors true 2024-09-12T01:32:57.377 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-12T01:32:57.377+0000 7f19f427e640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T01:32:57.386 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-12T01:32:57.386+0000 7f1a6b85e640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T01:32:57.387 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-12T01:32:57.388+0000 7f19ec09e640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:32:57.403 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-12T01:32:57.404+0000 7f1a6367e640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:32:58.381 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T01:32:58.403 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-12T01:32:58.403+0000 7f4232968640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T01:32:58.405 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-12T01:32:58.405+0000 7f7025f14640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T01:32:58.424 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-12T01:32:58.425+0000 7f701dd34640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:32:58.425 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-12T01:32:58.426+0000 7f422a788640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:32:58.477 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.477+0000 7f30a2bd8640 1 Processor -- start 2024-09-12T01:32:58.477 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.477+0000 7f30a2bd8640 1 -- start start 2024-09-12T01:32:58.478 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.478+0000 7f30a2bd8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f309c1619f0 0x7f309c0d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:58.479 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.478+0000 7f30a2bd8640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f309c0d0aa0 0x7f309c0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:58.479 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.478+0000 7f30a2bd8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f309c0d2f10 0x7f309c161f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:58.479 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.478+0000 7f30a2bd8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f309c0d1010 con 0x7f309c0d2f10 2024-09-12T01:32:58.479 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.478+0000 7f30a2bd8640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f309c0d12f0 con 0x7f309c0d0aa0 2024-09-12T01:32:58.479 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.478+0000 7f30a2bd8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f309c0d15d0 con 0x7f309c1619f0 2024-09-12T01:32:58.479 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.478+0000 7f30a1175640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f309c0d2f10 0x7f309c161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:58.479 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.478+0000 7f30a1175640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f309c0d2f10 0x7f309c161f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56648/0 (socket says 172.21.15.16:56648) 2024-09-12T01:32:58.479 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.478+0000 7f30a1175640 1 -- 172.21.15.16:0/580734851 learned_addr learned my addr 172.21.15.16:0/580734851 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:32:58.479 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.478+0000 7f309bfff640 1 --2- 172.21.15.16:0/580734851 >> v2:172.21.15.33:3300/0 conn(0x7f309c0d0aa0 0x7f309c0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:58.479 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.478+0000 7f30a0974640 1 --2- 172.21.15.16:0/580734851 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f309c1619f0 0x7f309c0d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:58.479 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.479+0000 7f30a1175640 1 -- 172.21.15.16:0/580734851 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f309c1619f0 msgr2=0x7f309c0d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:58.479 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.479+0000 7f30a1175640 1 --2- 172.21.15.16:0/580734851 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f309c1619f0 0x7f309c0d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:58.480 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.479+0000 7f30a1175640 1 -- 172.21.15.16:0/580734851 >> v2:172.21.15.33:3300/0 conn(0x7f309c0d0aa0 msgr2=0x7f309c0d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:58.480 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.479+0000 7f30a1175640 1 --2- 172.21.15.16:0/580734851 >> v2:172.21.15.33:3300/0 conn(0x7f309c0d0aa0 0x7f309c0d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:58.480 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.479+0000 7f30a1175640 1 -- 172.21.15.16:0/580734851 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309c0d18b0 con 0x7f309c0d2f10 2024-09-12T01:32:58.481 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.479+0000 7f30a1175640 1 --2- 172.21.15.16:0/580734851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c0d2f10 0x7f309c161f50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f309c0b3e20 tx=0x7f309402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26cfae6d7e88694 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:58.481 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.480+0000 7f309b7fe640 1 -- 172.21.15.16:0/580734851 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094037070 con 0x7f309c0d2f10 2024-09-12T01:32:58.481 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.480+0000 7f309b7fe640 1 -- 172.21.15.16:0/580734851 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3094032440 con 0x7f309c0d2f10 2024-09-12T01:32:58.481 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.480+0000 7f309b7fe640 1 -- 172.21.15.16:0/580734851 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094032810 con 0x7f309c0d2f10 2024-09-12T01:32:58.481 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.480+0000 7f30a2bd8640 1 -- 172.21.15.16:0/580734851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c0d2f10 msgr2=0x7f309c161f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:58.481 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.480+0000 7f30a2bd8640 1 --2- 172.21.15.16:0/580734851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c0d2f10 0x7f309c161f50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f309c0b3e20 tx=0x7f309402a140 comp rx=0 tx=0).stop 2024-09-12T01:32:58.481 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.480+0000 7f30a2bd8640 1 -- 172.21.15.16:0/580734851 shutdown_connections 2024-09-12T01:32:58.482 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.480+0000 7f30a2bd8640 1 --2- 172.21.15.16:0/580734851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c0d2f10 0x7f309c161f50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:58.482 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.480+0000 7f30a2bd8640 1 --2- 172.21.15.16:0/580734851 >> v2:172.21.15.33:3300/0 conn(0x7f309c0d0aa0 0x7f309c0d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:58.482 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.480+0000 7f30a2bd8640 1 --2- 172.21.15.16:0/580734851 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f309c1619f0 0x7f309c0d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:58.482 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.480+0000 7f30a2bd8640 1 -- 172.21.15.16:0/580734851 >> 172.21.15.16:0/580734851 conn(0x7f309c06c4c0 msgr2=0x7f309c09d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:58.482 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.480+0000 7f30a2bd8640 1 -- 172.21.15.16:0/580734851 shutdown_connections 2024-09-12T01:32:58.482 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.480+0000 7f30a2bd8640 1 -- 172.21.15.16:0/580734851 wait complete. 2024-09-12T01:32:58.482 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.482+0000 7f30a2bd8640 1 Processor -- start 2024-09-12T01:32:58.482 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.482+0000 7f30a2bd8640 1 -- start start 2024-09-12T01:32:58.483 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.482+0000 7f30a2bd8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f309c0d0aa0 0x7f309c0b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:58.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.482+0000 7f30a2bd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c0d2f10 0x7f309c1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:58.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a2bd8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f309c1619f0 0x7f309c158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:58.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a2bd8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f309c162610 con 0x7f309c0d2f10 2024-09-12T01:32:58.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a2bd8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f309c162490 con 0x7f309c1619f0 2024-09-12T01:32:58.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a2bd8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f309c162790 con 0x7f309c0d0aa0 2024-09-12T01:32:58.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a0974640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f309c0d0aa0 0x7f309c0b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:58.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a0974640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f309c0d0aa0 0x7f309c0b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:39894/0 (socket says 172.21.15.16:39894) 2024-09-12T01:32:58.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a0974640 1 -- 172.21.15.16:0/3481640304 learned_addr learned my addr 172.21.15.16:0/3481640304 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:32:58.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a1175640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f309c1619f0 0x7f309c158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:58.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f309bfff640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c0d2f10 0x7f309c1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:58.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a0974640 1 -- 172.21.15.16:0/3481640304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f309c1619f0 msgr2=0x7f309c158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:58.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a0974640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f309c1619f0 0x7f309c158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:58.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a0974640 1 -- 172.21.15.16:0/3481640304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c0d2f10 msgr2=0x7f309c1585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:58.485 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a0974640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c0d2f10 0x7f309c1585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:58.485 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a0974640 1 -- 172.21.15.16:0/3481640304 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309c0b8c10 con 0x7f309c0d0aa0 2024-09-12T01:32:58.485 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.483+0000 7f30a0974640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f309c0d0aa0 0x7f309c0b3fc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f308400e8a0 tx=0x7f308400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:58.503 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.503+0000 7f30997fa640 1 -- 172.21.15.16:0/3481640304 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084017070 con 0x7f309c0d0aa0 2024-09-12T01:32:58.503 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.503+0000 7f30997fa640 1 -- 172.21.15.16:0/3481640304 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f308400c940 con 0x7f309c0d0aa0 2024-09-12T01:32:58.503 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.503+0000 7f30997fa640 1 -- 172.21.15.16:0/3481640304 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308400cd30 con 0x7f309c0d0aa0 2024-09-12T01:32:58.503 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.503+0000 7f30a2bd8640 1 -- 172.21.15.16:0/3481640304 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309c0b8ef0 con 0x7f309c0d0aa0 2024-09-12T01:32:58.503 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.503+0000 7f30a2bd8640 1 -- 172.21.15.16:0/3481640304 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f309c0b94b0 con 0x7f309c0d0aa0 2024-09-12T01:32:58.503 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.504+0000 7f30997fa640 1 -- 172.21.15.16:0/3481640304 <== mon.2 v2:172.21.15.33:3301/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f308401a070 con 0x7f309c0d0aa0 2024-09-12T01:32:58.505 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.505+0000 7f30997fa640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f307c06bdf0 0x7f307c06e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:58.505 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.505+0000 7f30a2bd8640 1 -- 172.21.15.16:0/3481640304 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f309c162b90 con 0x7f309c0d0aa0 2024-09-12T01:32:58.505 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.505+0000 7f309bfff640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f307c06bdf0 0x7f307c06e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:58.505 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.505+0000 7f30997fa640 1 -- 172.21.15.16:0/3481640304 <== mon.2 v2:172.21.15.33:3301/0 5 ==== osd_map(13..13 src has 1..13) ==== 2791+0+0 (secure 0 0 0) 0x7f308408a320 con 0x7f309c0d0aa0 2024-09-12T01:32:58.505 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.505+0000 7f309bfff640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f307c06bdf0 0x7f307c06e2b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f309c0d0270 tx=0x7f308c00d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:58.509 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.510+0000 7f30997fa640 1 -- 172.21.15.16:0/3481640304 <== mon.2 v2:172.21.15.33:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f308405d630 con 0x7f309c0d0aa0 2024-09-12T01:32:58.687 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.688+0000 7f30a2bd8640 1 -- 172.21.15.16:0/3481640304 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f309c06cd30 con 0x7f309c0d0aa0 2024-09-12T01:32:58.890 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.891+0000 7f30997fa640 1 -- 172.21.15.16:0/3481640304 <== mon.2 v2:172.21.15.33:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+10866 (secure 0 0 0) 0x7f3084061e60 con 0x7f309c0d0aa0 2024-09-12T01:32:58.891 INFO:teuthology.misc.health.smithi016.stdout: 2024-09-12T01:32:58.891 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":14,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:32:58.512553+0000","last_up_change":"2024-09-12T01:32:58.512553+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":7.690000057220459,"score_stable":7.690000057220459,"optimal_score":0.12999999523162842,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:32:58.893 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.893+0000 7f30a2bd8640 1 -- 172.21.15.16:0/3481640304 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f307c06bdf0 msgr2=0x7f307c06e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:58.893 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.893+0000 7f30a2bd8640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f307c06bdf0 0x7f307c06e2b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f309c0d0270 tx=0x7f308c00d040 comp rx=0 tx=0).stop 2024-09-12T01:32:58.893 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.893+0000 7f30a2bd8640 1 -- 172.21.15.16:0/3481640304 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f309c0d0aa0 msgr2=0x7f309c0b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:58.893 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.893+0000 7f30a2bd8640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f309c0d0aa0 0x7f309c0b3fc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f308400e8a0 tx=0x7f308400ed60 comp rx=0 tx=0).stop 2024-09-12T01:32:58.893 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.893+0000 7f30a2bd8640 1 -- 172.21.15.16:0/3481640304 shutdown_connections 2024-09-12T01:32:58.893 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.893+0000 7f30a2bd8640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f307c06bdf0 0x7f307c06e2b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:58.893 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.893+0000 7f30a2bd8640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f309c1619f0 0x7f309c158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:58.894 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.893+0000 7f30a2bd8640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c0d2f10 0x7f309c1585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:58.894 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.893+0000 7f30a2bd8640 1 --2- 172.21.15.16:0/3481640304 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f309c0d0aa0 0x7f309c0b3fc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:58.894 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.893+0000 7f30a2bd8640 1 -- 172.21.15.16:0/3481640304 >> 172.21.15.16:0/3481640304 conn(0x7f309c06c4c0 msgr2=0x7f309c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:58.894 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.893+0000 7f30a2bd8640 1 -- 172.21.15.16:0/3481640304 shutdown_connections 2024-09-12T01:32:58.894 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:32:58.893+0000 7f30a2bd8640 1 -- 172.21.15.16:0/3481640304 wait complete. 2024-09-12T01:32:58.903 DEBUG:teuthology.misc:4 of 8 OSDs are up 2024-09-12T01:32:59.580 INFO:tasks.ceph.osd.6.smithi033.stderr:2024-09-12T01:32:59.579+0000 7f7ea51a0640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T01:32:59.596 INFO:tasks.ceph.osd.6.smithi033.stderr:2024-09-12T01:32:59.595+0000 7f7e9c7ac640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:32:59.626 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-12T01:32:59.625+0000 7f9173f7c640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T01:32:59.633 INFO:tasks.ceph.osd.7.smithi033.stderr:2024-09-12T01:32:59.632+0000 7f83acfd3640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T01:32:59.637 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-12T01:32:59.636+0000 7f1eb78c3640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T01:32:59.640 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-12T01:32:59.639+0000 7f916b588640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:32:59.648 INFO:tasks.ceph.osd.7.smithi033.stderr:2024-09-12T01:32:59.648+0000 7f83a4df3640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:32:59.651 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-12T01:32:59.651+0000 7f1eaf6e3640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:33:05.712 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T01:33:05.817 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.817+0000 7ff980077640 1 Processor -- start 2024-09-12T01:33:05.817 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.818+0000 7ff980077640 1 -- start start 2024-09-12T01:33:05.818 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.818+0000 7ff980077640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff97815e590 0x7ff9780ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:05.818 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.818+0000 7ff980077640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff9780cfcc0 0x7ff97806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:05.818 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.818+0000 7ff980077640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9780cee20 con 0x7ff97806cbd0 2024-09-12T01:33:05.818 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.818+0000 7ff980077640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7ff9780cf100 con 0x7ff9780cfcc0 2024-09-12T01:33:05.818 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.818+0000 7ff980077640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff9780cf3e0 con 0x7ff97815e590 2024-09-12T01:33:05.820 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.819+0000 7ff97d612640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff97815e590 0x7ff9780ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:05.820 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.819+0000 7ff97e614640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff9780cfcc0 0x7ff97806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:05.820 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.819+0000 7ff97d612640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff97815e590 0x7ff9780ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:39926/0 (socket says 172.21.15.16:39926) 2024-09-12T01:33:05.820 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.819+0000 7ff97d612640 1 -- 172.21.15.16:0/2504372085 learned_addr learned my addr 172.21.15.16:0/2504372085 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:05.820 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.819+0000 7ff97de13640 1 --1- 172.21.15.16:0/2504372085 >> v1:172.21.15.33:6789/0 conn(0x7ff97806cbd0 0x7ff97806cfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:51618/0 (socket says 172.21.15.16:51618) 2024-09-12T01:33:05.820 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.819+0000 7ff97ce11640 1 -- 172.21.15.16:0/2504372085 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3963505449 0 0) 0x7ff9780cee20 con 0x7ff97806cbd0 2024-09-12T01:33:05.820 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.819+0000 7ff97ce11640 1 -- 172.21.15.16:0/2504372085 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff960003520 con 0x7ff97806cbd0 2024-09-12T01:33:05.820 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.819+0000 7ff97e614640 1 -- 172.21.15.16:0/2504372085 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff97815e590 msgr2=0x7ff9780ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:05.820 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.819+0000 7ff97e614640 1 --2- 172.21.15.16:0/2504372085 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff97815e590 0x7ff9780ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:05.821 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.819+0000 7ff97e614640 1 -- 172.21.15.16:0/2504372085 >> v1:172.21.15.33:6789/0 conn(0x7ff97806cbd0 legacy=0x7ff97806cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:05.821 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.819+0000 7ff97e614640 1 -- 172.21.15.16:0/2504372085 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9780cf6c0 con 0x7ff9780cfcc0 2024-09-12T01:33:05.821 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.819+0000 7ff97d612640 1 --2- 172.21.15.16:0/2504372085 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff97815e590 0x7ff9780ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:05.821 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.820+0000 7ff97e614640 1 --2- 172.21.15.16:0/2504372085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9780cfcc0 0x7ff97806eb60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff9700097a0 tx=0x7ff97002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca01d071d233b501 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:05.821 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.820+0000 7ff97ce11640 1 -- 172.21.15.16:0/2504372085 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970037070 con 0x7ff9780cfcc0 2024-09-12T01:33:05.821 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.820+0000 7ff97ce11640 1 -- 172.21.15.16:0/2504372085 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff970032440 con 0x7ff9780cfcc0 2024-09-12T01:33:05.821 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.820+0000 7ff97ce11640 1 -- 172.21.15.16:0/2504372085 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700327f0 con 0x7ff9780cfcc0 2024-09-12T01:33:05.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.821+0000 7ff980077640 1 -- 172.21.15.16:0/2504372085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9780cfcc0 msgr2=0x7ff97806eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:05.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.821+0000 7ff980077640 1 --2- 172.21.15.16:0/2504372085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9780cfcc0 0x7ff97806eb60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff9700097a0 tx=0x7ff97002a180 comp rx=0 tx=0).stop 2024-09-12T01:33:05.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.821+0000 7ff980077640 1 -- 172.21.15.16:0/2504372085 shutdown_connections 2024-09-12T01:33:05.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.821+0000 7ff980077640 1 --2- 172.21.15.16:0/2504372085 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff97815e590 0x7ff9780ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:05.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.821+0000 7ff980077640 1 --2- 172.21.15.16:0/2504372085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9780cfcc0 0x7ff97806eb60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:05.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.821+0000 7ff980077640 1 -- 172.21.15.16:0/2504372085 >> 172.21.15.16:0/2504372085 conn(0x7ff978072520 msgr2=0x7ff978072900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:05.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.821+0000 7ff980077640 1 -- 172.21.15.16:0/2504372085 shutdown_connections 2024-09-12T01:33:05.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.821+0000 7ff980077640 1 -- 172.21.15.16:0/2504372085 wait complete. 2024-09-12T01:33:05.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.822+0000 7ff980077640 1 Processor -- start 2024-09-12T01:33:05.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.822+0000 7ff980077640 1 -- start start 2024-09-12T01:33:05.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.823+0000 7ff980077640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff97806cbd0 0x7ff97814e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:05.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.823+0000 7ff980077640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780cfcc0 0x7ff978155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:05.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.823+0000 7ff980077640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff97815e590 0x7ff9781555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:05.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.823+0000 7ff980077640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9780ce9e0 con 0x7ff9780cfcc0 2024-09-12T01:33:05.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.823+0000 7ff980077640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff9780ce860 con 0x7ff97806cbd0 2024-09-12T01:33:05.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.823+0000 7ff980077640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff9780ceb60 con 0x7ff97815e590 2024-09-12T01:33:05.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.823+0000 7ff97d612640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780cfcc0 0x7ff978155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:05.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.823+0000 7ff97d612640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780cfcc0 0x7ff978155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56688/0 (socket says 172.21.15.16:56688) 2024-09-12T01:33:05.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.823+0000 7ff97d612640 1 -- 172.21.15.16:0/1441952588 learned_addr learned my addr 172.21.15.16:0/1441952588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:05.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.823+0000 7ff97de13640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff97806cbd0 0x7ff97814e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:05.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.823+0000 7ff97e614640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff97815e590 0x7ff9781555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:05.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.824+0000 7ff97d612640 1 -- 172.21.15.16:0/1441952588 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff97815e590 msgr2=0x7ff9781555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:05.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.824+0000 7ff97d612640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff97815e590 0x7ff9781555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:05.826 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.824+0000 7ff97d612640 1 -- 172.21.15.16:0/1441952588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff97806cbd0 msgr2=0x7ff97814e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:05.826 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.824+0000 7ff97d612640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff97806cbd0 0x7ff97814e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:05.826 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.824+0000 7ff97d612640 1 -- 172.21.15.16:0/1441952588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97815f1b0 con 0x7ff9780cfcc0 2024-09-12T01:33:05.826 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.824+0000 7ff97d612640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780cfcc0 0x7ff978155080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00ea00 tx=0x7ff96c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:05.826 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.824+0000 7ff9767fc640 1 -- 172.21.15.16:0/1441952588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c017070 con 0x7ff9780cfcc0 2024-09-12T01:33:05.826 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.824+0000 7ff9767fc640 1 -- 172.21.15.16:0/1441952588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96c00ca60 con 0x7ff9780cfcc0 2024-09-12T01:33:05.826 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.824+0000 7ff9767fc640 1 -- 172.21.15.16:0/1441952588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c00ce10 con 0x7ff9780cfcc0 2024-09-12T01:33:05.826 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.824+0000 7ff980077640 1 -- 172.21.15.16:0/1441952588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97815f490 con 0x7ff9780cfcc0 2024-09-12T01:33:05.826 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.824+0000 7ff980077640 1 -- 172.21.15.16:0/1441952588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97815fa50 con 0x7ff9780cfcc0 2024-09-12T01:33:05.827 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.825+0000 7ff980077640 1 -- 172.21.15.16:0/1441952588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9780ae690 con 0x7ff9780cfcc0 2024-09-12T01:33:05.827 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.826+0000 7ff9767fc640 1 -- 172.21.15.16:0/1441952588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff96c01a070 con 0x7ff9780cfcc0 2024-09-12T01:33:05.827 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.826+0000 7ff9767fc640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff96406be30 0x7ff96406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:05.827 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.826+0000 7ff9767fc640 1 -- 172.21.15.16:0/1441952588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff96c08acf0 con 0x7ff9780cfcc0 2024-09-12T01:33:05.827 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.826+0000 7ff97de13640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff96406be30 0x7ff96406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:05.828 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.827+0000 7ff97de13640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff96406be30 0x7ff96406e2f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff968001560 tx=0x7ff968001660 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:05.830 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:05.831+0000 7ff9767fc640 1 -- 172.21.15.16:0/1441952588 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff96c08f070 con 0x7ff9780cfcc0 2024-09-12T01:33:06.010 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.010+0000 7ff980077640 1 -- 172.21.15.16:0/1441952588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff97815a190 con 0x7ff9780cfcc0 2024-09-12T01:33:06.010 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.011+0000 7ff9767fc640 1 -- 172.21.15.16:0/1441952588 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12873 (secure 0 0 0) 0x7ff96c059430 con 0x7ff9780cfcc0 2024-09-12T01:33:06.011 INFO:teuthology.misc.health.smithi016.stdout: 2024-09-12T01:33:06.011 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":17,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:33:01.726127+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:33:06.013 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.013+0000 7ff980077640 1 -- 172.21.15.16:0/1441952588 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff96406be30 msgr2=0x7ff96406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:06.013 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.013+0000 7ff980077640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff96406be30 0x7ff96406e2f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff968001560 tx=0x7ff968001660 comp rx=0 tx=0).stop 2024-09-12T01:33:06.013 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.013+0000 7ff980077640 1 -- 172.21.15.16:0/1441952588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780cfcc0 msgr2=0x7ff978155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:06.013 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.013+0000 7ff980077640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780cfcc0 0x7ff978155080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00ea00 tx=0x7ff96c00eec0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.013 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.013+0000 7ff980077640 1 -- 172.21.15.16:0/1441952588 shutdown_connections 2024-09-12T01:33:06.014 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.013+0000 7ff980077640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff96406be30 0x7ff96406e2f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.014 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.013+0000 7ff980077640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff97815e590 0x7ff9781555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.014 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.013+0000 7ff980077640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780cfcc0 0x7ff978155080 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.014 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.013+0000 7ff980077640 1 --2- 172.21.15.16:0/1441952588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff97806cbd0 0x7ff97814e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.014 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.013+0000 7ff980077640 1 -- 172.21.15.16:0/1441952588 >> 172.21.15.16:0/1441952588 conn(0x7ff978072520 msgr2=0x7ff9780ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:06.014 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.013+0000 7ff980077640 1 -- 172.21.15.16:0/1441952588 shutdown_connections 2024-09-12T01:33:06.014 INFO:teuthology.misc.health.smithi016.stderr:2024-09-12T01:33:06.013+0000 7ff980077640 1 -- 172.21.15.16:0/1441952588 wait complete. 2024-09-12T01:33:06.023 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-12T01:33:06.023 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-12T01:33:06.023 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T01:33:06.024 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T01:33:06.026 INFO:tasks.ceph.mds.a:Started 2024-09-12T01:33:06.026 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T01:33:06.026 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T01:33:06.028 INFO:tasks.ceph.mds.c:Started 2024-09-12T01:33:06.029 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T01:33:06.029 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T01:33:06.032 INFO:tasks.ceph.mds.b:Started 2024-09-12T01:33:06.032 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T01:33:06.032 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T01:33:06.035 INFO:tasks.ceph.mds.d:Started 2024-09-12T01:33:06.035 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-12T01:33:06.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:33:06.103 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-12T01:33:06.105 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-12T01:33:06.123 INFO:tasks.ceph.mds.b.smithi033.stdout:starting mds.b at 2024-09-12T01:33:06.123 INFO:tasks.ceph.mds.d.smithi033.stdout:starting mds.d at 2024-09-12T01:33:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.143+0000 7ff798949640 1 Processor -- start 2024-09-12T01:33:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.143+0000 7ff798949640 1 -- start start 2024-09-12T01:33:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff798949640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7940cdaf0 0x7ff7940cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff798949640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff794073620 0x7ff7940cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff798949640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff794073a80 0x7ff7940d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff798949640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff7940ce9e0 con 0x7ff794073620 2024-09-12T01:33:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff798949640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7ff7940cf080 con 0x7ff794073a80 2024-09-12T01:33:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff798949640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff7940ce410 con 0x7ff7940cdaf0 2024-09-12T01:33:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff791d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff794073620 0x7ff7940cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff791d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff794073620 0x7ff7940cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56722/0 (socket says 172.21.15.16:56722) 2024-09-12T01:33:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff791d9b640 1 -- 172.21.15.16:0/2148852600 learned_addr learned my addr 172.21.15.16:0/2148852600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff792d9d640 1 --2- 172.21.15.16:0/2148852600 >> v2:172.21.15.33:3300/0 conn(0x7ff794073a80 0x7ff7940d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff791d9b640 1 -- 172.21.15.16:0/2148852600 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7940cdaf0 msgr2=0x7ff7940cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:06.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff79259c640 1 --2- 172.21.15.16:0/2148852600 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7940cdaf0 0x7ff7940cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:06.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff791d9b640 1 --2- 172.21.15.16:0/2148852600 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7940cdaf0 0x7ff7940cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff791d9b640 1 -- 172.21.15.16:0/2148852600 >> v2:172.21.15.33:3300/0 conn(0x7ff794073a80 msgr2=0x7ff7940d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:06.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff791d9b640 1 --2- 172.21.15.16:0/2148852600 >> v2:172.21.15.33:3300/0 conn(0x7ff794073a80 0x7ff7940d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff791d9b640 1 -- 172.21.15.16:0/2148852600 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7940b43a0 con 0x7ff794073620 2024-09-12T01:33:06.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff792d9d640 1 --2- 172.21.15.16:0/2148852600 >> v2:172.21.15.33:3300/0 conn(0x7ff794073a80 0x7ff7940d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:06.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.144+0000 7ff79259c640 1 --2- 172.21.15.16:0/2148852600 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7940cdaf0 0x7ff7940cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:06.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.145+0000 7ff791d9b640 1 --2- 172.21.15.16:0/2148852600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff794073620 0x7ff7940cfb20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff7940cf600 tx=0x7ff78802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cf3dae447b9d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:06.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.145+0000 7ff79159a640 1 -- 172.21.15.16:0/2148852600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788037070 con 0x7ff794073620 2024-09-12T01:33:06.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.145+0000 7ff79159a640 1 -- 172.21.15.16:0/2148852600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff788032440 con 0x7ff794073620 2024-09-12T01:33:06.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.145+0000 7ff79159a640 1 -- 172.21.15.16:0/2148852600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788032810 con 0x7ff794073620 2024-09-12T01:33:06.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.146+0000 7ff798949640 1 -- 172.21.15.16:0/2148852600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff794073620 msgr2=0x7ff7940cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:06.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.146+0000 7ff798949640 1 --2- 172.21.15.16:0/2148852600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff794073620 0x7ff7940cfb20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff7940cf600 tx=0x7ff78802a1e0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.146+0000 7ff798949640 1 -- 172.21.15.16:0/2148852600 shutdown_connections 2024-09-12T01:33:06.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.146+0000 7ff798949640 1 --2- 172.21.15.16:0/2148852600 >> v2:172.21.15.33:3300/0 conn(0x7ff794073a80 0x7ff7940d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.146+0000 7ff798949640 1 --2- 172.21.15.16:0/2148852600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff794073620 0x7ff7940cfb20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.146+0000 7ff798949640 1 --2- 172.21.15.16:0/2148852600 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7940cdaf0 0x7ff7940cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.146+0000 7ff798949640 1 -- 172.21.15.16:0/2148852600 >> 172.21.15.16:0/2148852600 conn(0x7ff7940b2c50 msgr2=0x7ff7940b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:06.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.146+0000 7ff798949640 1 -- 172.21.15.16:0/2148852600 shutdown_connections 2024-09-12T01:33:06.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.146+0000 7ff798949640 1 -- 172.21.15.16:0/2148852600 wait complete. 2024-09-12T01:33:06.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.147+0000 7ff798949640 1 Processor -- start 2024-09-12T01:33:06.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.147+0000 7ff798949640 1 -- start start 2024-09-12T01:33:06.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.148+0000 7ff798949640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff794073620 0x7ff79415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:06.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.148+0000 7ff798949640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff794073a80 0x7ff79415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:06.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.148+0000 7ff798949640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cdaf0 0x7ff79413e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:06.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.148+0000 7ff798949640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7940b4800 con 0x7ff794073a80 2024-09-12T01:33:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.148+0000 7ff798949640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7940b4680 con 0x7ff7940cdaf0 2024-09-12T01:33:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.148+0000 7ff798949640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff794144de0 con 0x7ff794073620 2024-09-12T01:33:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.148+0000 7ff791d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff794073a80 0x7ff79415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.148+0000 7ff791d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff794073a80 0x7ff79415d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56738/0 (socket says 172.21.15.16:56738) 2024-09-12T01:33:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.148+0000 7ff791d9b640 1 -- 172.21.15.16:0/4092714909 learned_addr learned my addr 172.21.15.16:0/4092714909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.148+0000 7ff792d9d640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cdaf0 0x7ff79413e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.148+0000 7ff79259c640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff794073620 0x7ff79415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.149+0000 7ff791d9b640 1 -- 172.21.15.16:0/4092714909 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff794073620 msgr2=0x7ff79415a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.149+0000 7ff791d9b640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff794073620 0x7ff79415a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.149+0000 7ff791d9b640 1 -- 172.21.15.16:0/4092714909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cdaf0 msgr2=0x7ff79413e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.149+0000 7ff791d9b640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cdaf0 0x7ff79413e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.149+0000 7ff791d9b640 1 -- 172.21.15.16:0/4092714909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79415e3b0 con 0x7ff794073a80 2024-09-12T01:33:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.149+0000 7ff792d9d640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cdaf0 0x7ff79413e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:06.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.149+0000 7ff791d9b640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff794073a80 0x7ff79415d7c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff78802a220 tx=0x7ff7880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:06.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.149+0000 7ff782ffd640 1 -- 172.21.15.16:0/4092714909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788041020 con 0x7ff794073a80 2024-09-12T01:33:06.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.149+0000 7ff782ffd640 1 -- 172.21.15.16:0/4092714909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7880306c0 con 0x7ff794073a80 2024-09-12T01:33:06.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.149+0000 7ff798949640 1 -- 172.21.15.16:0/4092714909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79415e0d0 con 0x7ff794073a80 2024-09-12T01:33:06.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.149+0000 7ff798949640 1 -- 172.21.15.16:0/4092714909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff79413c380 con 0x7ff794073a80 2024-09-12T01:33:06.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.149+0000 7ff782ffd640 1 -- 172.21.15.16:0/4092714909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788037040 con 0x7ff794073a80 2024-09-12T01:33:06.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.150+0000 7ff798949640 1 -- 172.21.15.16:0/4092714909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7940b3f50 con 0x7ff794073a80 2024-09-12T01:33:06.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.151+0000 7ff782ffd640 1 -- 172.21.15.16:0/4092714909 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff788044020 con 0x7ff794073a80 2024-09-12T01:33:06.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.151+0000 7ff782ffd640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff77406c060 0x7ff77406e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:06.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.152+0000 7ff782ffd640 1 -- 172.21.15.16:0/4092714909 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff788030870 con 0x7ff794073a80 2024-09-12T01:33:06.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.152+0000 7ff79259c640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff77406c060 0x7ff77406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:06.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.152+0000 7ff79259c640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff77406c060 0x7ff77406e520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff7941e70c0 tx=0x7ff784008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:06.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.156+0000 7ff782ffd640 1 -- 172.21.15.16:0/4092714909 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff78804b030 con 0x7ff794073a80 2024-09-12T01:33:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.367+0000 7ff798949640 1 -- 172.21.15.16:0/4092714909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff79406c210 con 0x7ff794073a80 2024-09-12T01:33:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.367+0000 7ff782ffd640 1 -- 172.21.15.16:0/4092714909 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7ff788079610 con 0x7ff794073a80 2024-09-12T01:33:06.367 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:06.367 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"btime":"2024-09-12T01:33:06:108610+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4157213575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4157213575},{"type":"v1","addr":"172.21.15.16:6835","nonce":4157213575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T01:33:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-09-12T01:33:06.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.369+0000 7ff798949640 1 -- 172.21.15.16:0/4092714909 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff77406c060 msgr2=0x7ff77406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:06.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.369+0000 7ff798949640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff77406c060 0x7ff77406e520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff7941e70c0 tx=0x7ff784008040 comp rx=0 tx=0).stop 2024-09-12T01:33:06.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.369+0000 7ff798949640 1 -- 172.21.15.16:0/4092714909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff794073a80 msgr2=0x7ff79415d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:06.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.369+0000 7ff798949640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff794073a80 0x7ff79415d7c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff78802a220 tx=0x7ff7880046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.369+0000 7ff798949640 1 -- 172.21.15.16:0/4092714909 shutdown_connections 2024-09-12T01:33:06.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.369+0000 7ff798949640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff77406c060 0x7ff77406e520 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.369+0000 7ff798949640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cdaf0 0x7ff79413e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.369+0000 7ff798949640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff794073a80 0x7ff79415d7c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.369+0000 7ff798949640 1 --2- 172.21.15.16:0/4092714909 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff794073620 0x7ff79415a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:06.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.369+0000 7ff798949640 1 -- 172.21.15.16:0/4092714909 >> 172.21.15.16:0/4092714909 conn(0x7ff7940b2c50 msgr2=0x7ff7940d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:06.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.369+0000 7ff798949640 1 -- 172.21.15.16:0/4092714909 shutdown_connections 2024-09-12T01:33:06.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:06.369+0000 7ff798949640 1 -- 172.21.15.16:0/4092714909 wait complete. 2024-09-12T01:33:08.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:33:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.497+0000 7fc35dbd8640 1 Processor -- start 2024-09-12T01:33:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.497+0000 7fc35dbd8640 1 -- start start 2024-09-12T01:33:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.498+0000 7fc35dbd8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3581589a0 0x7fc358071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.498+0000 7fc35dbd8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc3580ae930 0x7fc358167ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.498+0000 7fc35dbd8640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc358158d80 con 0x7fc35815f580 2024-09-12T01:33:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.498+0000 7fc35dbd8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc35815fa90 con 0x7fc3581589a0 2024-09-12T01:33:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.498+0000 7fc35dbd8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fc358072830 con 0x7fc3580ae930 2024-09-12T01:33:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.498+0000 7fc3577fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3581589a0 0x7fc358071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.498+0000 7fc3577fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3581589a0 0x7fc358071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58078/0 (socket says 172.21.15.16:58078) 2024-09-12T01:33:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.498+0000 7fc3577fe640 1 -- 172.21.15.16:0/2606075693 learned_addr learned my addr 172.21.15.16:0/2606075693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.498+0000 7fc3577fe640 1 -- 172.21.15.16:0/2606075693 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc3580ae930 msgr2=0x7fc358167ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.498+0000 7fc357fff640 1 --2- 172.21.15.16:0/2606075693 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc3580ae930 0x7fc358167ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.499+0000 7fc3577fe640 1 --2- 172.21.15.16:0/2606075693 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc3580ae930 0x7fc358167ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.499+0000 7fc3577fe640 1 -- 172.21.15.16:0/2606075693 >> v1:172.21.15.33:6789/0 conn(0x7fc35815f580 legacy=0x7fc3580ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.499+0000 7fc3577fe640 1 -- 172.21.15.16:0/2606075693 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc358072b10 con 0x7fc3581589a0 2024-09-12T01:33:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.499+0000 7fc357fff640 1 --2- 172.21.15.16:0/2606075693 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc3580ae930 0x7fc358167ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.499+0000 7fc3577fe640 1 --2- 172.21.15.16:0/2606075693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3581589a0 0x7fc358071fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc348009870 tx=0x7fc34802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7b2f4722cfb04d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.499+0000 7fc356ffd640 1 -- 172.21.15.16:0/2606075693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348037070 con 0x7fc3581589a0 2024-09-12T01:33:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.499+0000 7fc356ffd640 1 -- 172.21.15.16:0/2606075693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc348032440 con 0x7fc3581589a0 2024-09-12T01:33:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.499+0000 7fc356ffd640 1 -- 172.21.15.16:0/2606075693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348032810 con 0x7fc3581589a0 2024-09-12T01:33:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.500+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2606075693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3581589a0 msgr2=0x7fc358071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.500+0000 7fc35dbd8640 1 --2- 172.21.15.16:0/2606075693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3581589a0 0x7fc358071fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc348009870 tx=0x7fc34802a1c0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.500+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2606075693 shutdown_connections 2024-09-12T01:33:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.500+0000 7fc35dbd8640 1 --2- 172.21.15.16:0/2606075693 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc3580ae930 0x7fc358167ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.500+0000 7fc35dbd8640 1 --2- 172.21.15.16:0/2606075693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3581589a0 0x7fc358071fe0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.500+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2606075693 >> 172.21.15.16:0/2606075693 conn(0x7fc3580d0470 msgr2=0x7fc3580d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.500+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2606075693 shutdown_connections 2024-09-12T01:33:08.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.500+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2606075693 wait complete. 2024-09-12T01:33:08.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.501+0000 7fc35dbd8640 1 Processor -- start 2024-09-12T01:33:08.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.502+0000 7fc35dbd8640 1 -- start start 2024-09-12T01:33:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.502+0000 7fc35dbd8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3580ae930 0x7fc35815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.502+0000 7fc35dbd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3581589a0 0x7fc3581f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.502+0000 7fc35dbd8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc35815f580 0x7fc3581f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.502+0000 7fc35dbd8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc358072520 con 0x7fc3581589a0 2024-09-12T01:33:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.502+0000 7fc35dbd8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3580730d0 con 0x7fc3580ae930 2024-09-12T01:33:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.502+0000 7fc35dbd8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fc358160f30 con 0x7fc35815f580 2024-09-12T01:33:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.502+0000 7fc34effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3581589a0 0x7fc3581f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc34effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3581589a0 0x7fc3581f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58082/0 (socket says 172.21.15.16:58082) 2024-09-12T01:33:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc34effd640 1 -- 172.21.15.16:0/2877262312 learned_addr learned my addr 172.21.15.16:0/2877262312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc3577fe640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3580ae930 0x7fc35815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc357fff640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc35815f580 0x7fc3581f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc34effd640 1 -- 172.21.15.16:0/2877262312 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc35815f580 msgr2=0x7fc3581f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc34effd640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc35815f580 0x7fc3581f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc34effd640 1 -- 172.21.15.16:0/2877262312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3580ae930 msgr2=0x7fc35815abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc34effd640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3580ae930 0x7fc35815abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc34effd640 1 -- 172.21.15.16:0/2877262312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc358159b40 con 0x7fc3581589a0 2024-09-12T01:33:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc34effd640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3581589a0 0x7fc3581f5300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc34000c3c0 tx=0x7fc34000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc354ff9640 1 -- 172.21.15.16:0/2877262312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3400052d0 con 0x7fc3581589a0 2024-09-12T01:33:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc354ff9640 1 -- 172.21.15.16:0/2877262312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc340005490 con 0x7fc3581589a0 2024-09-12T01:33:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2877262312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc358159e20 con 0x7fc3581589a0 2024-09-12T01:33:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc354ff9640 1 -- 172.21.15.16:0/2877262312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340007260 con 0x7fc3581589a0 2024-09-12T01:33:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.503+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2877262312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35815a3e0 con 0x7fc3581589a0 2024-09-12T01:33:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.504+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2877262312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc358071fe0 con 0x7fc3581589a0 2024-09-12T01:33:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.505+0000 7fc354ff9640 1 -- 172.21.15.16:0/2877262312 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc340007400 con 0x7fc3581589a0 2024-09-12T01:33:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.505+0000 7fc354ff9640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc33806be30 0x7fc33806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.505+0000 7fc354ff9640 1 -- 172.21.15.16:0/2877262312 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fc340089f20 con 0x7fc3581589a0 2024-09-12T01:33:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.505+0000 7fc3577fe640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc33806be30 0x7fc33806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.506+0000 7fc3577fe640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc33806be30 0x7fc33806e2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc348034000 tx=0x7fc34802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.510+0000 7fc354ff9640 1 -- 172.21.15.16:0/2877262312 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc3400585d0 con 0x7fc3581589a0 2024-09-12T01:33:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.719+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2877262312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc358160a40 con 0x7fc3581589a0 2024-09-12T01:33:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.719+0000 7fc354ff9640 1 -- 172.21.15.16:0/2877262312 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3903 (secure 0 0 0) 0x7fc34005c8d0 con 0x7fc3581589a0 2024-09-12T01:33:08.719 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:08.719 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"btime":"2024-09-12T01:33:07:110157+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4157213575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4157213575},{"type":"v1","addr":"172.21.15.16:6835","nonce":4157213575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1554674078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1554674078},{"type":"v1","addr":"172.21.15.16:6837","nonce":1554674078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/359404074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":359404074},{"type":"v1","addr":"172.21.15.33:6834","nonce":359404074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2179301157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2179301157},{"type":"v1","addr":"172.21.15.33:6835","nonce":2179301157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-12T01:33:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2024-09-12T01:33:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.721+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2877262312 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc33806be30 msgr2=0x7fc33806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.721+0000 7fc35dbd8640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc33806be30 0x7fc33806e2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc348034000 tx=0x7fc34802e000 comp rx=0 tx=0).stop 2024-09-12T01:33:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.721+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2877262312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3581589a0 msgr2=0x7fc3581f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.721+0000 7fc35dbd8640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3581589a0 0x7fc3581f5300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc34000c3c0 tx=0x7fc34000c880 comp rx=0 tx=0).stop 2024-09-12T01:33:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.721+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2877262312 shutdown_connections 2024-09-12T01:33:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.721+0000 7fc35dbd8640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc33806be30 0x7fc33806e2f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.721+0000 7fc35dbd8640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc35815f580 0x7fc3581f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.721+0000 7fc35dbd8640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3581589a0 0x7fc3581f5300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.721+0000 7fc35dbd8640 1 --2- 172.21.15.16:0/2877262312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3580ae930 0x7fc35815abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.721+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2877262312 >> 172.21.15.16:0/2877262312 conn(0x7fc3580d0470 msgr2=0x7fc35815e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:08.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.721+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2877262312 shutdown_connections 2024-09-12T01:33:08.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.721+0000 7fc35dbd8640 1 -- 172.21.15.16:0/2877262312 wait complete. 2024-09-12T01:33:08.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T01:33:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.833+0000 7ff094e65640 1 Processor -- start 2024-09-12T01:33:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.833+0000 7ff094e65640 1 -- start start 2024-09-12T01:33:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.834+0000 7ff094e65640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff09015e560 0x7ff0900ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.834+0000 7ff094e65640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0900d1550 0x7ff0900b7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.834+0000 7ff094e65640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0900d3480 con 0x7ff0900cf510 2024-09-12T01:33:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.834+0000 7ff094e65640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff0900d3760 con 0x7ff0900d1550 2024-09-12T01:33:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.834+0000 7ff094e65640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff0900d3a40 con 0x7ff09015e560 2024-09-12T01:33:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.834+0000 7ff08ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0900d1550 0x7ff0900b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.834+0000 7ff08dd9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff0900cf510 0x7ff0900d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58988/0 (socket says 172.21.15.16:58988) 2024-09-12T01:33:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.834+0000 7ff08dd9b640 1 -- 172.21.15.16:0/166028415 learned_addr learned my addr 172.21.15.16:0/166028415 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.834+0000 7ff08e59c640 1 --2- 172.21.15.16:0/166028415 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff09015e560 0x7ff0900ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.835+0000 7ff08ed9d640 1 -- 172.21.15.16:0/166028415 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff09015e560 msgr2=0x7ff0900ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.835+0000 7ff08ed9d640 1 --2- 172.21.15.16:0/166028415 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff09015e560 0x7ff0900ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.835+0000 7ff08ed9d640 1 -- 172.21.15.16:0/166028415 >> v1:172.21.15.16:6789/0 conn(0x7ff0900cf510 legacy=0x7ff0900d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:08.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.835+0000 7ff08ed9d640 1 -- 172.21.15.16:0/166028415 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0900d3d20 con 0x7ff0900d1550 2024-09-12T01:33:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.835+0000 7ff08ed9d640 1 --2- 172.21.15.16:0/166028415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0900d1550 0x7ff0900b7790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff0880097a0 tx=0x7ff08802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b667ed1d68636b52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.836+0000 7ff08d59a640 1 -- 172.21.15.16:0/166028415 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088037070 con 0x7ff0900d1550 2024-09-12T01:33:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.836+0000 7ff08d59a640 1 -- 172.21.15.16:0/166028415 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff088032440 con 0x7ff0900d1550 2024-09-12T01:33:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.836+0000 7ff08d59a640 1 -- 172.21.15.16:0/166028415 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088032810 con 0x7ff0900d1550 2024-09-12T01:33:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.836+0000 7ff094e65640 1 -- 172.21.15.16:0/166028415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0900d1550 msgr2=0x7ff0900b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.836+0000 7ff094e65640 1 --2- 172.21.15.16:0/166028415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0900d1550 0x7ff0900b7790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff0880097a0 tx=0x7ff08802a180 comp rx=0 tx=0).stop 2024-09-12T01:33:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.836+0000 7ff094e65640 1 -- 172.21.15.16:0/166028415 shutdown_connections 2024-09-12T01:33:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.836+0000 7ff094e65640 1 --2- 172.21.15.16:0/166028415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0900d1550 0x7ff0900b7790 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.836+0000 7ff094e65640 1 --2- 172.21.15.16:0/166028415 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff09015e560 0x7ff0900ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.836+0000 7ff094e65640 1 -- 172.21.15.16:0/166028415 >> 172.21.15.16:0/166028415 conn(0x7ff090073850 msgr2=0x7ff09006cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.836+0000 7ff094e65640 1 -- 172.21.15.16:0/166028415 shutdown_connections 2024-09-12T01:33:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.836+0000 7ff094e65640 1 -- 172.21.15.16:0/166028415 wait complete. 2024-09-12T01:33:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.838+0000 7ff094e65640 1 Processor -- start 2024-09-12T01:33:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.838+0000 7ff094e65640 1 -- start start 2024-09-12T01:33:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.838+0000 7ff094e65640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0900cf510 0x7ff0900ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.838+0000 7ff094e65640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff0900d1550 0x7ff0900cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.838+0000 7ff094e65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09015e560 0x7ff0900ad300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.838+0000 7ff094e65640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff09015eac0 con 0x7ff09015e560 2024-09-12T01:33:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.838+0000 7ff094e65640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff09015e940 con 0x7ff0900cf510 2024-09-12T01:33:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.838+0000 7ff094e65640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff0900cf950 con 0x7ff0900d1550 2024-09-12T01:33:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.838+0000 7ff08ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09015e560 0x7ff0900ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.838+0000 7ff08ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09015e560 0x7ff0900ad300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58094/0 (socket says 172.21.15.16:58094) 2024-09-12T01:33:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.838+0000 7ff08ed9d640 1 -- 172.21.15.16:0/3217778862 learned_addr learned my addr 172.21.15.16:0/3217778862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.839+0000 7ff08e59c640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0900cf510 0x7ff0900ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.839+0000 7ff08dd9b640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff0900d1550 0x7ff0900cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.839+0000 7ff08ed9d640 1 -- 172.21.15.16:0/3217778862 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff0900d1550 msgr2=0x7ff0900cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.839+0000 7ff08ed9d640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff0900d1550 0x7ff0900cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.839+0000 7ff08ed9d640 1 -- 172.21.15.16:0/3217778862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0900cf510 msgr2=0x7ff0900ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.839+0000 7ff08ed9d640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0900cf510 0x7ff0900ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.839+0000 7ff08ed9d640 1 -- 172.21.15.16:0/3217778862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0900aadd0 con 0x7ff09015e560 2024-09-12T01:33:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.839+0000 7ff08ed9d640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09015e560 0x7ff0900ad300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff0880098c0 tx=0x7ff0880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.839+0000 7ff086ffd640 1 -- 172.21.15.16:0/3217778862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088037040 con 0x7ff09015e560 2024-09-12T01:33:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.839+0000 7ff086ffd640 1 -- 172.21.15.16:0/3217778862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff088042070 con 0x7ff09015e560 2024-09-12T01:33:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.839+0000 7ff094e65640 1 -- 172.21.15.16:0/3217778862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0900ccd20 con 0x7ff09015e560 2024-09-12T01:33:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.839+0000 7ff086ffd640 1 -- 172.21.15.16:0/3217778862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0880308a0 con 0x7ff09015e560 2024-09-12T01:33:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.839+0000 7ff094e65640 1 -- 172.21.15.16:0/3217778862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0900cd2e0 con 0x7ff09015e560 2024-09-12T01:33:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.840+0000 7ff094e65640 1 -- 172.21.15.16:0/3217778862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0900ceee0 con 0x7ff09015e560 2024-09-12T01:33:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.841+0000 7ff086ffd640 1 -- 172.21.15.16:0/3217778862 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff088030a40 con 0x7ff09015e560 2024-09-12T01:33:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.841+0000 7ff086ffd640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff07006be30 0x7ff07006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.841+0000 7ff086ffd640 1 -- 172.21.15.16:0/3217778862 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff0880ab270 con 0x7ff09015e560 2024-09-12T01:33:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.841+0000 7ff08e59c640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff07006be30 0x7ff07006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.842+0000 7ff08e59c640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff07006be30 0x7ff07006e2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff080011340 tx=0x7ff08000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:08.845+0000 7ff086ffd640 1 -- 172.21.15.16:0/3217778862 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff0880799b0 con 0x7ff09015e560 2024-09-12T01:33:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.022+0000 7ff094e65640 1 -- 172.21.15.16:0/3217778862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff0900b8240 con 0x7ff09015e560 2024-09-12T01:33:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.022+0000 7ff086ffd640 1 -- 172.21.15.16:0/3217778862 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7ff08807dcb0 con 0x7ff09015e560 2024-09-12T01:33:09.022 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:09.022 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-12T01:33:09.022 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:33:09.022 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-12T01:33:09.022 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-12T01:33:09.022 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:33:09.023 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-12T01:33:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.024+0000 7ff094e65640 1 -- 172.21.15.16:0/3217778862 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff07006be30 msgr2=0x7ff07006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.024+0000 7ff094e65640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff07006be30 0x7ff07006e2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff080011340 tx=0x7ff08000b040 comp rx=0 tx=0).stop 2024-09-12T01:33:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.024+0000 7ff094e65640 1 -- 172.21.15.16:0/3217778862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09015e560 msgr2=0x7ff0900ad300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.024+0000 7ff094e65640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09015e560 0x7ff0900ad300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff0880098c0 tx=0x7ff0880046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.024+0000 7ff094e65640 1 -- 172.21.15.16:0/3217778862 shutdown_connections 2024-09-12T01:33:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.024+0000 7ff094e65640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff07006be30 0x7ff07006e2f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.024+0000 7ff094e65640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09015e560 0x7ff0900ad300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.024+0000 7ff094e65640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff0900d1550 0x7ff0900cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.024+0000 7ff094e65640 1 --2- 172.21.15.16:0/3217778862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0900cf510 0x7ff0900ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.024+0000 7ff094e65640 1 -- 172.21.15.16:0/3217778862 >> 172.21.15.16:0/3217778862 conn(0x7ff090073850 msgr2=0x7ff09015b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.024+0000 7ff094e65640 1 -- 172.21.15.16:0/3217778862 shutdown_connections 2024-09-12T01:33:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.024+0000 7ff094e65640 1 -- 172.21.15.16:0/3217778862 wait complete. 2024-09-12T01:33:09.035 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T01:33:09.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T01:33:09.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.141+0000 7f5527ffd640 1 Processor -- start 2024-09-12T01:33:09.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.141+0000 7f5527ffd640 1 -- start start 2024-09-12T01:33:09.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.142+0000 7f5527ffd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5520162130 0x7f5520162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:09.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.142+0000 7f5527ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55200b9a80 0x7f55200b9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.142+0000 7f5527ffd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55200d39d0 con 0x7f552015e770 2024-09-12T01:33:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.142+0000 7f5527ffd640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f55200b7df0 con 0x7f55200b9a80 2024-09-12T01:33:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.142+0000 7f5527ffd640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f5520162ab0 con 0x7f5520162130 2024-09-12T01:33:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.142+0000 7f5525d99640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f552015e770 0x7f55200d32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58996/0 (socket says 172.21.15.16:58996) 2024-09-12T01:33:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.142+0000 7f5525d99640 1 -- 172.21.15.16:0/734982800 learned_addr learned my addr 172.21.15.16:0/734982800 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.142+0000 7f552659a640 1 --2- 172.21.15.16:0/734982800 >> v2:172.21.15.33:3300/0 conn(0x7f55200b9a80 0x7f55200b9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5525598640 1 --2- 172.21.15.16:0/734982800 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5520162130 0x7f5520162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1500911190 0 0) 0x7f55200d39d0 con 0x7f552015e770 2024-09-12T01:33:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5508003520 con 0x7f552015e770 2024-09-12T01:33:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2800728440 0 0) 0x7f5508003520 con 0x7f552015e770 2024-09-12T01:33:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5520162130 msgr2=0x7f5520162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 --2- 172.21.15.16:0/734982800 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5520162130 0x7f5520162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:09.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 >> v2:172.21.15.33:3300/0 conn(0x7f55200b9a80 msgr2=0x7f55200b9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:09.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 --2- 172.21.15.16:0/734982800 >> v2:172.21.15.33:3300/0 conn(0x7f55200b9a80 0x7f55200b9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:09.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55200b74c0 con 0x7f552015e770 2024-09-12T01:33:09.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f5518001670 con 0x7f552015e770 2024-09-12T01:33:09.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 >> v1:172.21.15.16:6789/0 conn(0x7f552015e770 legacy=0x7f55200d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:09.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 --2- 172.21.15.16:0/734982800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55080039b0 0x7f5508003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:09.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 --2- 172.21.15.16:0/734982800 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5520162130 0x7f5508004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:09.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 --2- 172.21.15.16:0/734982800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508004d40 0x7f55080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:09.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55200b7df0 con 0x7f5508004d40 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5520162ab0 con 0x7f55080039b0 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f550800b9c0 con 0x7f5520162130 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f552659a640 1 --2- 172.21.15.16:0/734982800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508004d40 0x7f55080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5525d99640 1 --2- 172.21.15.16:0/734982800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55080039b0 0x7f5508003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f5525598640 1 --2- 172.21.15.16:0/734982800 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5520162130 0x7f5508004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f552659a640 1 -- 172.21.15.16:0/734982800 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5520162130 msgr2=0x7f5508004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f552659a640 1 --2- 172.21.15.16:0/734982800 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5520162130 0x7f5508004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f552659a640 1 -- 172.21.15.16:0/734982800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55080039b0 msgr2=0x7f5508003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f552659a640 1 --2- 172.21.15.16:0/734982800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55080039b0 0x7f5508003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.143+0000 7f552659a640 1 -- 172.21.15.16:0/734982800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5508003520 con 0x7f5508004d40 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.144+0000 7f552659a640 1 --2- 172.21.15.16:0/734982800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508004d40 0x7f55080051c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5510009fc0 tx=0x7f5508005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2d944c2c13886a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.144+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510009c10 con 0x7f5508004d40 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.144+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5510036070 con 0x7f5508004d40 2024-09-12T01:33:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.144+0000 7f5524d97640 1 -- 172.21.15.16:0/734982800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551002d070 con 0x7f5508004d40 2024-09-12T01:33:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.145+0000 7f5527ffd640 1 -- 172.21.15.16:0/734982800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508004d40 msgr2=0x7f55080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.145+0000 7f5527ffd640 1 --2- 172.21.15.16:0/734982800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508004d40 0x7f55080051c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5510009fc0 tx=0x7f5508005830 comp rx=0 tx=0).stop 2024-09-12T01:33:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.145+0000 7f552659a640 1 -- 172.21.15.16:0/734982800 reap_dead start 2024-09-12T01:33:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.145+0000 7f5527ffd640 1 -- 172.21.15.16:0/734982800 shutdown_connections 2024-09-12T01:33:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.145+0000 7f5527ffd640 1 -- 172.21.15.16:0/734982800 >> 172.21.15.16:0/734982800 conn(0x7f552009d660 msgr2=0x7f55200d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.145+0000 7f5527ffd640 1 -- 172.21.15.16:0/734982800 shutdown_connections 2024-09-12T01:33:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.145+0000 7f5527ffd640 1 -- 172.21.15.16:0/734982800 wait complete. 2024-09-12T01:33:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.146+0000 7f5527ffd640 1 Processor -- start 2024-09-12T01:33:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.146+0000 7f5527ffd640 1 -- start start 2024-09-12T01:33:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5527ffd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5508004d40 0x7f55200a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5527ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200b6390 0x7f552009e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5527ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55200b6810 0x7f552009e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5527ffd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55200d51c0 con 0x7f55200b6390 2024-09-12T01:33:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5527ffd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55200b7a80 con 0x7f55200b6810 2024-09-12T01:33:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5527ffd640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f55200d5340 con 0x7f5508004d40 2024-09-12T01:33:09.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5525598640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200b6390 0x7f552009e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:09.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5525598640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200b6390 0x7f552009e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58118/0 (socket says 172.21.15.16:58118) 2024-09-12T01:33:09.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5525598640 1 -- 172.21.15.16:0/3915578152 learned_addr learned my addr 172.21.15.16:0/3915578152 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:09.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f552659a640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55200b6810 0x7f552009e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:09.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5525d99640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5508004d40 0x7f55200a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:09.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5525598640 1 -- 172.21.15.16:0/3915578152 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5508004d40 msgr2=0x7f55200a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:09.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5525598640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5508004d40 0x7f55200a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:09.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5525598640 1 -- 172.21.15.16:0/3915578152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55200b6810 msgr2=0x7f552009e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:09.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5525598640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55200b6810 0x7f552009e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:09.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5525598640 1 -- 172.21.15.16:0/3915578152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55200ca540 con 0x7f55200b6390 2024-09-12T01:33:09.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f552659a640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55200b6810 0x7f552009e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:09.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5525598640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200b6390 0x7f552009e140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5514011910 tx=0x7f5514011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:09.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f551e7fc640 1 -- 172.21.15.16:0/3915578152 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551400faf0 con 0x7f55200b6390 2024-09-12T01:33:09.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f551e7fc640 1 -- 172.21.15.16:0/3915578152 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f551400fc90 con 0x7f55200b6390 2024-09-12T01:33:09.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f5527ffd640 1 -- 172.21.15.16:0/3915578152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55200ca820 con 0x7f55200b6390 2024-09-12T01:33:09.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.147+0000 7f551e7fc640 1 -- 172.21.15.16:0/3915578152 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514014440 con 0x7f55200b6390 2024-09-12T01:33:09.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.148+0000 7f5527ffd640 1 -- 172.21.15.16:0/3915578152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55200c81a0 con 0x7f55200b6390 2024-09-12T01:33:09.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.148+0000 7f5527ffd640 1 -- 172.21.15.16:0/3915578152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5520069110 con 0x7f55200b6390 2024-09-12T01:33:09.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.149+0000 7f551e7fc640 1 -- 172.21.15.16:0/3915578152 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5514012040 con 0x7f55200b6390 2024-09-12T01:33:09.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.149+0000 7f551e7fc640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f550c06be30 0x7f550c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:09.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.149+0000 7f551e7fc640 1 -- 172.21.15.16:0/3915578152 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f551408b220 con 0x7f55200b6390 2024-09-12T01:33:09.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.149+0000 7f5525d99640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f550c06be30 0x7f550c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:09.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.150+0000 7f5525d99640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f550c06be30 0x7f550c06e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f55201611a0 tx=0x7f5518009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:09.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.154+0000 7f551e7fc640 1 -- 172.21.15.16:0/3915578152 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5514019030 con 0x7f55200b6390 2024-09-12T01:33:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:09.333+0000 7f5527ffd640 1 -- 172.21.15.16:0/3915578152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f55200c7fd0 con 0x7f55200b6390 2024-09-12T01:33:10.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.121+0000 7f551e7fc640 1 -- 172.21.15.16:0/3915578152 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f5514059f20 con 0x7f55200b6390 2024-09-12T01:33:10.121 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-12T01:33:10.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.123+0000 7f5527ffd640 1 -- 172.21.15.16:0/3915578152 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f550c06be30 msgr2=0x7f550c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.123+0000 7f5527ffd640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f550c06be30 0x7f550c06e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f55201611a0 tx=0x7f5518009040 comp rx=0 tx=0).stop 2024-09-12T01:33:10.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.123+0000 7f5527ffd640 1 -- 172.21.15.16:0/3915578152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200b6390 msgr2=0x7f552009e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.123+0000 7f5527ffd640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200b6390 0x7f552009e140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5514011910 tx=0x7f5514011dd0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.124+0000 7f5527ffd640 1 -- 172.21.15.16:0/3915578152 shutdown_connections 2024-09-12T01:33:10.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.124+0000 7f5527ffd640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f550c06be30 0x7f550c06e2f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.124+0000 7f5527ffd640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55200b6810 0x7f552009e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.124+0000 7f5527ffd640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200b6390 0x7f552009e140 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.124+0000 7f5527ffd640 1 --2- 172.21.15.16:0/3915578152 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5508004d40 0x7f55200a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.124+0000 7f5527ffd640 1 -- 172.21.15.16:0/3915578152 >> 172.21.15.16:0/3915578152 conn(0x7f552009d660 msgr2=0x7f55200b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:10.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.124+0000 7f5527ffd640 1 -- 172.21.15.16:0/3915578152 shutdown_connections 2024-09-12T01:33:10.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.124+0000 7f5527ffd640 1 -- 172.21.15.16:0/3915578152 wait complete. 2024-09-12T01:33:10.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-12T01:33:10.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.258+0000 7f2f7d247640 1 Processor -- start 2024-09-12T01:33:10.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.258+0000 7f2f7d247640 1 -- start start 2024-09-12T01:33:10.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.259+0000 7f2f7d247640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2f780c5c10 0x7f2f7815df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.259+0000 7f2f7d247640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f7815cea0 con 0x7f2f780cf700 2024-09-12T01:33:10.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.259+0000 7f2f7d247640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f7815d180 con 0x7f2f780cfae0 2024-09-12T01:33:10.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.259+0000 7f2f7d247640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f2f7815d460 con 0x7f2f780c5c10 2024-09-12T01:33:10.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.259+0000 7f2f76d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2f780cf700 0x7f2f7815c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59006/0 (socket says 172.21.15.16:59006) 2024-09-12T01:33:10.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.259+0000 7f2f76d9d640 1 -- 172.21.15.16:0/2468434970 learned_addr learned my addr 172.21.15.16:0/2468434970 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:10.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.259+0000 7f2f7759e640 1 --2- 172.21.15.16:0/2468434970 >> v2:172.21.15.33:3300/0 conn(0x7f2f780c5c10 0x7f2f7815df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.259+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 298720071 0 0) 0x7f2f7815cea0 con 0x7f2f780cf700 2024-09-12T01:33:10.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.259+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f64003520 con 0x7f2f780cf700 2024-09-12T01:33:10.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.259+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 312357687 0 0) 0x7f2f7815d180 con 0x7f2f780cfae0 2024-09-12T01:33:10.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.259+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f7815cea0 con 0x7f2f780cfae0 2024-09-12T01:33:10.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 907551894 0 0) 0x7f2f64003520 con 0x7f2f780cf700 2024-09-12T01:33:10.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 >> v2:172.21.15.33:3300/0 conn(0x7f2f780c5c10 msgr2=0x7f2f7815df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:10.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f75d9b640 1 --2- 172.21.15.16:0/2468434970 >> v2:172.21.15.33:3300/0 conn(0x7f2f780c5c10 0x7f2f7815df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 >> v1:172.21.15.33:6789/0 conn(0x7f2f780cfae0 legacy=0x7f2f780c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7815da40 con 0x7f2f780cf700 2024-09-12T01:33:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f2f68001670 con 0x7f2f780cf700 2024-09-12T01:33:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 >> v1:172.21.15.16:6789/0 conn(0x7f2f780cf700 legacy=0x7f2f7815c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f75d9b640 1 --2- 172.21.15.16:0/2468434970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f64003b40 0x7f2f64003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f75d9b640 1 --2- 172.21.15.16:0/2468434970 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2f64004800 0x7f2f64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f75d9b640 1 --2- 172.21.15.16:0/2468434970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f640051c0 0x7f2f6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7815d460 con 0x7f2f64003b40 2024-09-12T01:33:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6400c040 con 0x7f2f640051c0 2024-09-12T01:33:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6400c240 con 0x7f2f64004800 2024-09-12T01:33:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f76d9d640 1 --2- 172.21.15.16:0/2468434970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f64003b40 0x7f2f64003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f7759e640 1 --2- 172.21.15.16:0/2468434970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f640051c0 0x7f2f6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f7659c640 1 --2- 172.21.15.16:0/2468434970 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2f64004800 0x7f2f64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f76d9d640 1 -- 172.21.15.16:0/2468434970 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2f64004800 msgr2=0x7f2f64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f76d9d640 1 --2- 172.21.15.16:0/2468434970 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2f64004800 0x7f2f64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f76d9d640 1 -- 172.21.15.16:0/2468434970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f640051c0 msgr2=0x7f2f6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f76d9d640 1 --2- 172.21.15.16:0/2468434970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f640051c0 0x7f2f6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f76d9d640 1 -- 172.21.15.16:0/2468434970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f64003520 con 0x7f2f64003b40 2024-09-12T01:33:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.260+0000 7f2f7759e640 1 -- 172.21.15.16:0/2468434970 reap_dead start 2024-09-12T01:33:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.261+0000 7f2f76d9d640 1 --2- 172.21.15.16:0/2468434970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f64003b40 0x7f2f64003f40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800c190 tx=0x7f2f64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7623bf3ff134c9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.261+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68007c40 con 0x7f2f64003b40 2024-09-12T01:33:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.261+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f68005ad0 con 0x7f2f64003b40 2024-09-12T01:33:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.261+0000 7f2f75d9b640 1 -- 172.21.15.16:0/2468434970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68007070 con 0x7f2f64003b40 2024-09-12T01:33:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.261+0000 7f2f7d247640 1 -- 172.21.15.16:0/2468434970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f64003b40 msgr2=0x7f2f64003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.261+0000 7f2f7d247640 1 --2- 172.21.15.16:0/2468434970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f64003b40 0x7f2f64003f40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800c190 tx=0x7f2f64005830 comp rx=0 tx=0).stop 2024-09-12T01:33:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.262+0000 7f2f7d247640 1 -- 172.21.15.16:0/2468434970 shutdown_connections 2024-09-12T01:33:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.262+0000 7f2f7d247640 1 --2- 172.21.15.16:0/2468434970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f64003b40 0x7f2f64003f40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.262+0000 7f2f7d247640 1 -- 172.21.15.16:0/2468434970 >> 172.21.15.16:0/2468434970 conn(0x7f2f7806c560 msgr2=0x7f2f780b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.262+0000 7f2f7d247640 1 -- 172.21.15.16:0/2468434970 shutdown_connections 2024-09-12T01:33:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.262+0000 7f2f7d247640 1 -- 172.21.15.16:0/2468434970 wait complete. 2024-09-12T01:33:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.263+0000 7f2f7d247640 1 Processor -- start 2024-09-12T01:33:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.263+0000 7f2f7d247640 1 -- start start 2024-09-12T01:33:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7d247640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f64003b40 0x7f2f781f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7d247640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f780c8140 0x7f2f78159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7d247640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2f780cbb30 0x7f2f78144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7d247640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7815ca00 con 0x7f2f780c8140 2024-09-12T01:33:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7d247640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7815c880 con 0x7f2f64003b40 2024-09-12T01:33:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7d247640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7815cb80 con 0x7f2f780cbb30 2024-09-12T01:33:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f780c8140 0x7f2f78159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f780c8140 0x7f2f78159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58134/0 (socket says 172.21.15.16:58134) 2024-09-12T01:33:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7659c640 1 -- 172.21.15.16:0/1149118349 learned_addr learned my addr 172.21.15.16:0/1149118349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f76d9d640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f64003b40 0x7f2f781f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7759e640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2f780cbb30 0x7f2f78144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7659c640 1 -- 172.21.15.16:0/1149118349 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2f780cbb30 msgr2=0x7f2f78144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7659c640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2f780cbb30 0x7f2f78144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7659c640 1 -- 172.21.15.16:0/1149118349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f64003b40 msgr2=0x7f2f781f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7659c640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f64003b40 0x7f2f781f1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.264+0000 7f2f7659c640 1 -- 172.21.15.16:0/1149118349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f780c8dd0 con 0x7f2f780c8140 2024-09-12T01:33:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.265+0000 7f2f7659c640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f780c8140 0x7f2f78159150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000f7a0 tx=0x7f2f6000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.265+0000 7f2f6f7fe640 1 -- 172.21.15.16:0/1149118349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6001a070 con 0x7f2f780c8140 2024-09-12T01:33:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.265+0000 7f2f6f7fe640 1 -- 172.21.15.16:0/1149118349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f6000fe50 con 0x7f2f780c8140 2024-09-12T01:33:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.265+0000 7f2f7d247640 1 -- 172.21.15.16:0/1149118349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f780c90b0 con 0x7f2f780c8140 2024-09-12T01:33:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.265+0000 7f2f6f7fe640 1 -- 172.21.15.16:0/1149118349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60007050 con 0x7f2f780c8140 2024-09-12T01:33:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.265+0000 7f2f7d247640 1 -- 172.21.15.16:0/1149118349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f780c9950 con 0x7f2f780c8140 2024-09-12T01:33:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.266+0000 7f2f7d247640 1 -- 172.21.15.16:0/1149118349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f780c5500 con 0x7f2f780c8140 2024-09-12T01:33:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.266+0000 7f2f6f7fe640 1 -- 172.21.15.16:0/1149118349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f6001d070 con 0x7f2f780c8140 2024-09-12T01:33:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.267+0000 7f2f6f7fe640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2f5806be30 0x7f2f5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.267+0000 7f2f6f7fe640 1 -- 172.21.15.16:0/1149118349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f2f6008a400 con 0x7f2f780c8140 2024-09-12T01:33:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.267+0000 7f2f76d9d640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2f5806be30 0x7f2f5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.267+0000 7f2f76d9d640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2f5806be30 0x7f2f5806e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2f68032000 tx=0x7f2f68007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.271+0000 7f2f6f7fe640 1 -- 172.21.15.16:0/1149118349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2f60058940 con 0x7f2f780c8140 2024-09-12T01:33:10.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:10.446+0000 7f2f7d247640 1 -- 172.21.15.16:0/1149118349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f2f780c9b20 con 0x7f2f780c8140 2024-09-12T01:33:11.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.123+0000 7f2f6f7fe640 1 -- 172.21.15.16:0/1149118349 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7f2f6005cc40 con 0x7f2f780c8140 2024-09-12T01:33:11.123 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-12T01:33:11.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.126+0000 7f2f7d247640 1 -- 172.21.15.16:0/1149118349 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2f5806be30 msgr2=0x7f2f5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:11.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.126+0000 7f2f7d247640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2f5806be30 0x7f2f5806e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2f68032000 tx=0x7f2f68007000 comp rx=0 tx=0).stop 2024-09-12T01:33:11.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.127+0000 7f2f7d247640 1 -- 172.21.15.16:0/1149118349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f780c8140 msgr2=0x7f2f78159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:11.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.127+0000 7f2f7d247640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f780c8140 0x7f2f78159150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000f7a0 tx=0x7f2f6000fc60 comp rx=0 tx=0).stop 2024-09-12T01:33:11.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.127+0000 7f2f7d247640 1 -- 172.21.15.16:0/1149118349 shutdown_connections 2024-09-12T01:33:11.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.127+0000 7f2f7d247640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2f5806be30 0x7f2f5806e2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:11.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.127+0000 7f2f7d247640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2f780cbb30 0x7f2f78144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:11.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.127+0000 7f2f7d247640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f780c8140 0x7f2f78159150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:11.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.127+0000 7f2f7d247640 1 --2- 172.21.15.16:0/1149118349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f64003b40 0x7f2f781f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:11.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.127+0000 7f2f7d247640 1 -- 172.21.15.16:0/1149118349 >> 172.21.15.16:0/1149118349 conn(0x7f2f7806c560 msgr2=0x7f2f780b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:11.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.127+0000 7f2f7d247640 1 -- 172.21.15.16:0/1149118349 shutdown_connections 2024-09-12T01:33:11.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.127+0000 7f2f7d247640 1 -- 172.21.15.16:0/1149118349 wait complete. 2024-09-12T01:33:11.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T01:33:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.264+0000 7f96051d8640 1 Processor -- start 2024-09-12T01:33:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.264+0000 7f96051d8640 1 -- start start 2024-09-12T01:33:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f96051d8640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f96000d0090 0x7f96000d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f96051d8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f96000d0e30 0x7f96000d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f96051d8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96000cf050 con 0x7f960015e6a0 2024-09-12T01:33:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f96051d8640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f96000cfc00 con 0x7f96000d0090 2024-09-12T01:33:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f96051d8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f96000cf2d0 con 0x7f96000d0e30 2024-09-12T01:33:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f95fed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f960015e6a0 0x7f96000d4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59010/0 (socket says 172.21.15.16:59010) 2024-09-12T01:33:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f95fed9d640 1 -- 172.21.15.16:0/3909538784 learned_addr learned my addr 172.21.15.16:0/3909538784 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f95ff59e640 1 --2- 172.21.15.16:0/3909538784 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f96000d0e30 0x7f96000d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f95f7fff640 1 --2- 172.21.15.16:0/3909538784 >> v2:172.21.15.33:3300/0 conn(0x7f96000d0090 0x7f96000d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2148859963 0 0) 0x7f95f0001290 con 0x7f960015e6a0 2024-09-12T01:33:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95ec003520 con 0x7f960015e6a0 2024-09-12T01:33:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2504607057 0 0) 0x7f95ec003520 con 0x7f960015e6a0 2024-09-12T01:33:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f96000d0e30 msgr2=0x7f96000d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f95fe59c640 1 --2- 172.21.15.16:0/3909538784 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f96000d0e30 0x7f96000d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 >> v2:172.21.15.33:3300/0 conn(0x7f96000d0090 msgr2=0x7f96000d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f95fe59c640 1 --2- 172.21.15.16:0/3909538784 >> v2:172.21.15.33:3300/0 conn(0x7f96000d0090 0x7f96000d3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96000cf5b0 con 0x7f960015e6a0 2024-09-12T01:33:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.265+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f95f00019e0 con 0x7f960015e6a0 2024-09-12T01:33:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 >> v1:172.21.15.16:6789/0 conn(0x7f960015e6a0 legacy=0x7f96000d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95fe59c640 1 --2- 172.21.15.16:0/3909538784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95ec0039b0 0x7f95ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95fe59c640 1 --2- 172.21.15.16:0/3909538784 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f96000d0e30 0x7f95ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95fe59c640 1 --2- 172.21.15.16:0/3909538784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95ec004d40 0x7f95ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96000cfc00 con 0x7f95ec0039b0 2024-09-12T01:33:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96000cf2d0 con 0x7f95ec004d40 2024-09-12T01:33:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f95ec00b9c0 con 0x7f96000d0e30 2024-09-12T01:33:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95fed9d640 1 --2- 172.21.15.16:0/3909538784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95ec0039b0 0x7f95ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95f7fff640 1 --2- 172.21.15.16:0/3909538784 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f96000d0e30 0x7f95ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95ff59e640 1 --2- 172.21.15.16:0/3909538784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95ec004d40 0x7f95ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95fed9d640 1 -- 172.21.15.16:0/3909538784 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f96000d0e30 msgr2=0x7f95ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95fed9d640 1 --2- 172.21.15.16:0/3909538784 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f96000d0e30 0x7f95ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95fed9d640 1 -- 172.21.15.16:0/3909538784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95ec004d40 msgr2=0x7f95ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95fed9d640 1 --2- 172.21.15.16:0/3909538784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95ec004d40 0x7f95ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.266+0000 7f95fed9d640 1 -- 172.21.15.16:0/3909538784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ec003520 con 0x7f95ec0039b0 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.267+0000 7f95fed9d640 1 --2- 172.21.15.16:0/3909538784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95ec0039b0 0x7f95ec003dd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f95f000c490 tx=0x7f95ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33683b8b329a50db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.267+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f000c0e0 con 0x7f95ec0039b0 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.267+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f0034070 con 0x7f95ec0039b0 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.267+0000 7f95fe59c640 1 -- 172.21.15.16:0/3909538784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f00074d0 con 0x7f95ec0039b0 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.267+0000 7f96051d8640 1 -- 172.21.15.16:0/3909538784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95ec0039b0 msgr2=0x7f95ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.267+0000 7f96051d8640 1 --2- 172.21.15.16:0/3909538784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95ec0039b0 0x7f95ec003dd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f95f000c490 tx=0x7f95ec005830 comp rx=0 tx=0).stop 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.267+0000 7f95ff59e640 1 -- 172.21.15.16:0/3909538784 reap_dead start 2024-09-12T01:33:11.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.267+0000 7f96051d8640 1 -- 172.21.15.16:0/3909538784 shutdown_connections 2024-09-12T01:33:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.267+0000 7f96051d8640 1 -- 172.21.15.16:0/3909538784 >> 172.21.15.16:0/3909538784 conn(0x7f960009d730 msgr2=0x7f96000d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.268+0000 7f96051d8640 1 -- 172.21.15.16:0/3909538784 shutdown_connections 2024-09-12T01:33:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.268+0000 7f96051d8640 1 -- 172.21.15.16:0/3909538784 wait complete. 2024-09-12T01:33:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.269+0000 7f96051d8640 1 Processor -- start 2024-09-12T01:33:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.269+0000 7f96051d8640 1 -- start start 2024-09-12T01:33:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f96051d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95ec0039b0 0x7f96000ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f96051d8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f960015d8b0 0x7f96000a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f96051d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96000a1410 0x7f96001e88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f96051d8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96000d1440 con 0x7f96000a1410 2024-09-12T01:33:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f96051d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96000d12c0 con 0x7f95ec0039b0 2024-09-12T01:33:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f96051d8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f96000d15c0 con 0x7f960015d8b0 2024-09-12T01:33:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95ff59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96000a1410 0x7f96001e88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95ff59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96000a1410 0x7f96001e88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58162/0 (socket says 172.21.15.16:58162) 2024-09-12T01:33:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95ff59e640 1 -- 172.21.15.16:0/395606400 learned_addr learned my addr 172.21.15.16:0/395606400 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95fed9d640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95ec0039b0 0x7f96000ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95f7fff640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f960015d8b0 0x7f96000a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95ff59e640 1 -- 172.21.15.16:0/395606400 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f960015d8b0 msgr2=0x7f96000a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95ff59e640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f960015d8b0 0x7f96000a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95ff59e640 1 -- 172.21.15.16:0/395606400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95ec0039b0 msgr2=0x7f96000ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95ff59e640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95ec0039b0 0x7f96000ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95ff59e640 1 -- 172.21.15.16:0/395606400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96000a1e90 con 0x7f96000a1410 2024-09-12T01:33:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95ff59e640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96000a1410 0x7f96001e88d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f96000b7010 tx=0x7f95f800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95f77fe640 1 -- 172.21.15.16:0/395606400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f800fc80 con 0x7f96000a1410 2024-09-12T01:33:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95f77fe640 1 -- 172.21.15.16:0/395606400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f800fe20 con 0x7f96000a1410 2024-09-12T01:33:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f95f77fe640 1 -- 172.21.15.16:0/395606400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8015670 con 0x7f96000a1410 2024-09-12T01:33:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.270+0000 7f96051d8640 1 -- 172.21.15.16:0/395606400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96000a18d0 con 0x7f96000a1410 2024-09-12T01:33:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.271+0000 7f96051d8640 1 -- 172.21.15.16:0/395606400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960015c380 con 0x7f96000a1410 2024-09-12T01:33:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.271+0000 7f96051d8640 1 -- 172.21.15.16:0/395606400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96000d3160 con 0x7f96000a1410 2024-09-12T01:33:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.272+0000 7f95f77fe640 1 -- 172.21.15.16:0/395606400 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95f8015810 con 0x7f96000a1410 2024-09-12T01:33:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.272+0000 7f95f77fe640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f95e006be30 0x7f95e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.272+0000 7f95f77fe640 1 -- 172.21.15.16:0/395606400 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f95f808b9c0 con 0x7f96000a1410 2024-09-12T01:33:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.273+0000 7f95fed9d640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f95e006be30 0x7f95e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.273+0000 7f95fed9d640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f95e006be30 0x7f95e006e2f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f95f0033000 tx=0x7f95f002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:11.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.277+0000 7f95f77fe640 1 -- 172.21.15.16:0/395606400 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f95f808de20 con 0x7f96000a1410 2024-09-12T01:33:11.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:11.491+0000 7f96051d8640 1 -- 172.21.15.16:0/395606400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f960006da60 con 0x7f96000a1410 2024-09-12T01:33:12.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.148+0000 7f95f77fe640 1 -- 172.21.15.16:0/395606400 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f95f8019030 con 0x7f96000a1410 2024-09-12T01:33:12.148 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T01:33:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.153+0000 7f96051d8640 1 -- 172.21.15.16:0/395606400 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f95e006be30 msgr2=0x7f95e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.153+0000 7f96051d8640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f95e006be30 0x7f95e006e2f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f95f0033000 tx=0x7f95f002e000 comp rx=0 tx=0).stop 2024-09-12T01:33:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.153+0000 7f96051d8640 1 -- 172.21.15.16:0/395606400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96000a1410 msgr2=0x7f96001e88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.153+0000 7f96051d8640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96000a1410 0x7f96001e88d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f96000b7010 tx=0x7f95f800f1b0 comp rx=0 tx=0).stop 2024-09-12T01:33:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.153+0000 7f96051d8640 1 -- 172.21.15.16:0/395606400 shutdown_connections 2024-09-12T01:33:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.153+0000 7f96051d8640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f95e006be30 0x7f95e006e2f0 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f95f0033000 tx=0x7f95f002e000 comp rx=0 tx=0).stop 2024-09-12T01:33:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.153+0000 7f96051d8640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96000a1410 0x7f96001e88d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.153+0000 7f96051d8640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f960015d8b0 0x7f96000a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.153+0000 7f96051d8640 1 --2- 172.21.15.16:0/395606400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95ec0039b0 0x7f96000ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.153+0000 7f96051d8640 1 -- 172.21.15.16:0/395606400 >> 172.21.15.16:0/395606400 conn(0x7f960009d730 msgr2=0x7f96000b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.154+0000 7f96051d8640 1 -- 172.21.15.16:0/395606400 shutdown_connections 2024-09-12T01:33:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.154+0000 7f96051d8640 1 -- 172.21.15.16:0/395606400 wait complete. 2024-09-12T01:33:12.182 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-12T01:33:12.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-12T01:33:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.296+0000 7fe731850640 1 Processor -- start 2024-09-12T01:33:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.296+0000 7fe731850640 1 -- start start 2024-09-12T01:33:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe731850640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe72c15e4d0 0x7fe72c0d3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe731850640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe72c0d1fb0 con 0x7fe72c0cfa90 2024-09-12T01:33:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe731850640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe72c0d2290 con 0x7fe72c0d13a0 2024-09-12T01:33:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe731850640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fe72c0d2570 con 0x7fe72c15e4d0 2024-09-12T01:33:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe72a7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe72c0cfa90 0x7fe72c0d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59026/0 (socket says 172.21.15.16:59026) 2024-09-12T01:33:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe72a7fc640 1 -- 172.21.15.16:0/362538160 learned_addr learned my addr 172.21.15.16:0/362538160 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe72affd640 1 --2- 172.21.15.16:0/362538160 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe72c15e4d0 0x7fe72c0d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 198520615 0 0) 0x7fe72c0d1fb0 con 0x7fe72c0cfa90 2024-09-12T01:33:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe714003520 con 0x7fe72c0cfa90 2024-09-12T01:33:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2684696772 0 0) 0x7fe72c0d2290 con 0x7fe72c0d13a0 2024-09-12T01:33:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe72c0d1fb0 con 0x7fe72c0d13a0 2024-09-12T01:33:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2114273326 0 0) 0x7fe714003520 con 0x7fe72c0cfa90 2024-09-12T01:33:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe72c15e4d0 msgr2=0x7fe72c0d3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 --2- 172.21.15.16:0/362538160 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe72c15e4d0 0x7fe72c0d3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 >> v1:172.21.15.33:6789/0 conn(0x7fe72c0d13a0 legacy=0x7fe72c0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c0d2850 con 0x7fe72c0cfa90 2024-09-12T01:33:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7fe720001670 con 0x7fe72c0cfa90 2024-09-12T01:33:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 >> v1:172.21.15.16:6789/0 conn(0x7fe72c0cfa90 legacy=0x7fe72c0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 --2- 172.21.15.16:0/362538160 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe72c15e4d0 0x7fe714003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 --2- 172.21.15.16:0/362538160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7140040d0 0x7fe714004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 --2- 172.21.15.16:0/362538160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714004d40 0x7fe7140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c0d2570 con 0x7fe7140040d0 2024-09-12T01:33:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe71400bb50 con 0x7fe714004d40 2024-09-12T01:33:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fe71400bd20 con 0x7fe72c15e4d0 2024-09-12T01:33:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.297+0000 7fe72a7fc640 1 --2- 172.21.15.16:0/362538160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7140040d0 0x7fe714004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.298+0000 7fe72affd640 1 --2- 172.21.15.16:0/362538160 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe72c15e4d0 0x7fe714003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.298+0000 7fe72b7fe640 1 --2- 172.21.15.16:0/362538160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714004d40 0x7fe7140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.298+0000 7fe72a7fc640 1 -- 172.21.15.16:0/362538160 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe72c15e4d0 msgr2=0x7fe714003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.298+0000 7fe72a7fc640 1 --2- 172.21.15.16:0/362538160 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe72c15e4d0 0x7fe714003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.298+0000 7fe72a7fc640 1 -- 172.21.15.16:0/362538160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714004d40 msgr2=0x7fe7140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.298+0000 7fe72a7fc640 1 --2- 172.21.15.16:0/362538160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714004d40 0x7fe7140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.298+0000 7fe72a7fc640 1 -- 172.21.15.16:0/362538160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe714003520 con 0x7fe7140040d0 2024-09-12T01:33:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.298+0000 7fe72b7fe640 1 --2- 172.21.15.16:0/362538160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714004d40 0x7fe7140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.298+0000 7fe72a7fc640 1 --2- 172.21.15.16:0/362538160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7140040d0 0x7fe714004800 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe72000c190 tx=0x7fe714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b1910b9a8f82df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.299+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720007cc0 con 0x7fe7140040d0 2024-09-12T01:33:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.299+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe720005af0 con 0x7fe7140040d0 2024-09-12T01:33:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.299+0000 7fe729ffb640 1 -- 172.21.15.16:0/362538160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720007070 con 0x7fe7140040d0 2024-09-12T01:33:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.299+0000 7fe731850640 1 -- 172.21.15.16:0/362538160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7140040d0 msgr2=0x7fe714004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.299+0000 7fe731850640 1 --2- 172.21.15.16:0/362538160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7140040d0 0x7fe714004800 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe72000c190 tx=0x7fe714005830 comp rx=0 tx=0).stop 2024-09-12T01:33:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.299+0000 7fe72b7fe640 1 -- 172.21.15.16:0/362538160 reap_dead start 2024-09-12T01:33:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.299+0000 7fe731850640 1 -- 172.21.15.16:0/362538160 shutdown_connections 2024-09-12T01:33:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.299+0000 7fe731850640 1 -- 172.21.15.16:0/362538160 >> 172.21.15.16:0/362538160 conn(0x7fe72c09d650 msgr2=0x7fe72c0d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.299+0000 7fe731850640 1 -- 172.21.15.16:0/362538160 shutdown_connections 2024-09-12T01:33:12.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.299+0000 7fe731850640 1 -- 172.21.15.16:0/362538160 wait complete. 2024-09-12T01:33:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.300+0000 7fe731850640 1 Processor -- start 2024-09-12T01:33:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.301+0000 7fe731850640 1 -- start start 2024-09-12T01:33:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.301+0000 7fe731850640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe7140040d0 0x7fe72c0b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.301+0000 7fe731850640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c0ccf10 0x7fe72c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.301+0000 7fe731850640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe72c1e5940 0x7fe72c1e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.301+0000 7fe731850640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c15e8b0 con 0x7fe72c0ccf10 2024-09-12T01:33:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.301+0000 7fe731850640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c0d4d80 con 0x7fe72c1e5940 2024-09-12T01:33:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.301+0000 7fe731850640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fe72c15ea30 con 0x7fe7140040d0 2024-09-12T01:33:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe72a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c0ccf10 0x7fe72c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe72a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c0ccf10 0x7fe72c0cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58172/0 (socket says 172.21.15.16:58172) 2024-09-12T01:33:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe72a7fc640 1 -- 172.21.15.16:0/3615468817 learned_addr learned my addr 172.21.15.16:0/3615468817 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe72affd640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe7140040d0 0x7fe72c0b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe72b7fe640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe72c1e5940 0x7fe72c1e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe72a7fc640 1 -- 172.21.15.16:0/3615468817 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe7140040d0 msgr2=0x7fe72c0b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe72a7fc640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe7140040d0 0x7fe72c0b74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe72a7fc640 1 -- 172.21.15.16:0/3615468817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe72c1e5940 msgr2=0x7fe72c1e5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe72a7fc640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe72c1e5940 0x7fe72c1e5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe72a7fc640 1 -- 172.21.15.16:0/3615468817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c0cd9d0 con 0x7fe72c0ccf10 2024-09-12T01:33:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe72a7fc640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c0ccf10 0x7fe72c0cd390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe72000c190 tx=0x7fe72c0d2180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe713fff640 1 -- 172.21.15.16:0/3615468817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720001670 con 0x7fe72c0ccf10 2024-09-12T01:33:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe713fff640 1 -- 172.21.15.16:0/3615468817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe720042070 con 0x7fe72c0ccf10 2024-09-12T01:33:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe731850640 1 -- 172.21.15.16:0/3615468817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72c0cdcb0 con 0x7fe72c0ccf10 2024-09-12T01:33:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe713fff640 1 -- 172.21.15.16:0/3615468817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7200072b0 con 0x7fe72c0ccf10 2024-09-12T01:33:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.302+0000 7fe731850640 1 -- 172.21.15.16:0/3615468817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72c14c730 con 0x7fe72c0ccf10 2024-09-12T01:33:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.303+0000 7fe731850640 1 -- 172.21.15.16:0/3615468817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe72c069110 con 0x7fe72c0ccf10 2024-09-12T01:33:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.304+0000 7fe713fff640 1 -- 172.21.15.16:0/3615468817 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe720005970 con 0x7fe72c0ccf10 2024-09-12T01:33:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.304+0000 7fe713fff640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe71806be60 0x7fe71806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.304+0000 7fe713fff640 1 -- 172.21.15.16:0/3615468817 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe7200aad60 con 0x7fe72c0ccf10 2024-09-12T01:33:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.304+0000 7fe72affd640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe71806be60 0x7fe71806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.305+0000 7fe72affd640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe71806be60 0x7fe71806e320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009ae0 tx=0x7fe71c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.309+0000 7fe713fff640 1 -- 172.21.15.16:0/3615468817 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe720079710 con 0x7fe72c0ccf10 2024-09-12T01:33:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:12.493+0000 7fe731850640 1 -- 172.21.15.16:0/3615468817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fe72c14c240 con 0x7fe72c0ccf10 2024-09-12T01:33:12.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:33:12.681 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:33:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.149+0000 7fe713fff640 1 -- 172.21.15.16:0/3615468817 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7fe720079010 con 0x7fe72c0ccf10 2024-09-12T01:33:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.153+0000 7fe731850640 1 -- 172.21.15.16:0/3615468817 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe71806be60 msgr2=0x7fe71806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.153+0000 7fe731850640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe71806be60 0x7fe71806e320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009ae0 tx=0x7fe71c009190 comp rx=0 tx=0).stop 2024-09-12T01:33:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.153+0000 7fe731850640 1 -- 172.21.15.16:0/3615468817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c0ccf10 msgr2=0x7fe72c0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.153+0000 7fe731850640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c0ccf10 0x7fe72c0cd390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe72000c190 tx=0x7fe72c0d2180 comp rx=0 tx=0).stop 2024-09-12T01:33:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.153+0000 7fe731850640 1 -- 172.21.15.16:0/3615468817 shutdown_connections 2024-09-12T01:33:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.153+0000 7fe731850640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe71806be60 0x7fe71806e320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.153+0000 7fe731850640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe72c1e5940 0x7fe72c1e5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.153+0000 7fe731850640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c0ccf10 0x7fe72c0cd390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.153+0000 7fe731850640 1 --2- 172.21.15.16:0/3615468817 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe7140040d0 0x7fe72c0b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:13.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.153+0000 7fe731850640 1 -- 172.21.15.16:0/3615468817 >> 172.21.15.16:0/3615468817 conn(0x7fe72c09d650 msgr2=0x7fe72c0b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:13.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.153+0000 7fe731850640 1 -- 172.21.15.16:0/3615468817 shutdown_connections 2024-09-12T01:33:13.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.153+0000 7fe731850640 1 -- 172.21.15.16:0/3615468817 wait complete. 2024-09-12T01:33:13.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T01:33:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.284+0000 7f7f0479f640 1 Processor -- start 2024-09-12T01:33:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.284+0000 7f7f0479f640 1 -- start start 2024-09-12T01:33:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.285+0000 7f7f0479f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7efc0b6660 0x7f7efc0d4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.285+0000 7f7f0479f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7efc15f4e0 0x7f7efc0d35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.285+0000 7f7f0479f640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7efc0d5350 con 0x7f7efc15f100 2024-09-12T01:33:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.285+0000 7f7f0479f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7efc0d5120 con 0x7f7efc0b6660 2024-09-12T01:33:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.285+0000 7f7f0479f640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f7efc0d3b20 con 0x7f7efc15f4e0 2024-09-12T01:33:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.285+0000 7f7f01d3a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7efc0b6660 0x7f7efc0d4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.285+0000 7f7f01d3a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7efc0b6660 0x7f7efc0d4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58174/0 (socket says 172.21.15.16:58174) 2024-09-12T01:33:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.285+0000 7f7f01d3a640 1 -- 172.21.15.16:0/2001239870 learned_addr learned my addr 172.21.15.16:0/2001239870 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.285+0000 7f7f02d3c640 1 --2- 172.21.15.16:0/2001239870 >> v2:172.21.15.33:3300/0 conn(0x7f7efc15f4e0 0x7f7efc0d35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.285+0000 7f7f01d3a640 1 -- 172.21.15.16:0/2001239870 >> v2:172.21.15.33:3300/0 conn(0x7f7efc15f4e0 msgr2=0x7f7efc0d35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.285+0000 7f7f01d3a640 1 --2- 172.21.15.16:0/2001239870 >> v2:172.21.15.33:3300/0 conn(0x7f7efc15f4e0 0x7f7efc0d35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.285+0000 7f7f01d3a640 1 -- 172.21.15.16:0/2001239870 >> v1:172.21.15.33:6789/0 conn(0x7f7efc15f100 legacy=0x7f7efc0b5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.285+0000 7f7f01d3a640 1 -- 172.21.15.16:0/2001239870 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc0d2820 con 0x7f7efc0b6660 2024-09-12T01:33:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.286+0000 7f7f01d3a640 1 --2- 172.21.15.16:0/2001239870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc0b6660 0x7f7efc0d4be0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0d43d0 tx=0x7f7eec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2231c10d58813f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.286+0000 7f7f01539640 1 -- 172.21.15.16:0/2001239870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec037070 con 0x7f7efc0b6660 2024-09-12T01:33:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.286+0000 7f7f01539640 1 -- 172.21.15.16:0/2001239870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec032440 con 0x7f7efc0b6660 2024-09-12T01:33:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.286+0000 7f7f01539640 1 -- 172.21.15.16:0/2001239870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec032810 con 0x7f7efc0b6660 2024-09-12T01:33:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.287+0000 7f7f0479f640 1 -- 172.21.15.16:0/2001239870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc0b6660 msgr2=0x7f7efc0d4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.287+0000 7f7f0479f640 1 --2- 172.21.15.16:0/2001239870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc0b6660 0x7f7efc0d4be0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0d43d0 tx=0x7f7eec02a1e0 comp rx=0 tx=0).stop 2024-09-12T01:33:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.287+0000 7f7f0479f640 1 -- 172.21.15.16:0/2001239870 shutdown_connections 2024-09-12T01:33:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.287+0000 7f7f0479f640 1 --2- 172.21.15.16:0/2001239870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc0b6660 0x7f7efc0d4be0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.287+0000 7f7f0479f640 1 --2- 172.21.15.16:0/2001239870 >> v2:172.21.15.33:3300/0 conn(0x7f7efc15f4e0 0x7f7efc0d35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.287+0000 7f7f0479f640 1 -- 172.21.15.16:0/2001239870 >> 172.21.15.16:0/2001239870 conn(0x7f7efc09d6a0 msgr2=0x7f7efc0d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.287+0000 7f7f0479f640 1 -- 172.21.15.16:0/2001239870 shutdown_connections 2024-09-12T01:33:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.287+0000 7f7f0479f640 1 -- 172.21.15.16:0/2001239870 wait complete. 2024-09-12T01:33:13.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.288+0000 7f7f0479f640 1 Processor -- start 2024-09-12T01:33:13.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.288+0000 7f7f0479f640 1 -- start start 2024-09-12T01:33:13.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f0479f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7efc0b6660 0x7f7efc1f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:13.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f0479f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc15f100 0x7f7efc0ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:13.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f0479f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f7efc15f4e0 0x7f7efc15b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:13.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f0479f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc0d30c0 con 0x7f7efc15f100 2024-09-12T01:33:13.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f0479f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc0d2de0 con 0x7f7efc0b6660 2024-09-12T01:33:13.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f0479f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f7efc15f990 con 0x7f7efc15f4e0 2024-09-12T01:33:13.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f01d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc15f100 0x7f7efc0ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:13.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f01d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc15f100 0x7f7efc0ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58176/0 (socket says 172.21.15.16:58176) 2024-09-12T01:33:13.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f01d3a640 1 -- 172.21.15.16:0/439897399 learned_addr learned my addr 172.21.15.16:0/439897399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:13.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f0253b640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7efc0b6660 0x7f7efc1f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:13.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f02d3c640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f7efc15f4e0 0x7f7efc15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:13.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f01d3a640 1 -- 172.21.15.16:0/439897399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f7efc15f4e0 msgr2=0x7f7efc15b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:13.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f01d3a640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f7efc15f4e0 0x7f7efc15b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:13.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f01d3a640 1 -- 172.21.15.16:0/439897399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7efc0b6660 msgr2=0x7f7efc1f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:13.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f01d3a640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7efc0b6660 0x7f7efc1f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:13.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f01d3a640 1 -- 172.21.15.16:0/439897399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc1f7e20 con 0x7f7efc15f100 2024-09-12T01:33:13.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f0253b640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7efc0b6660 0x7f7efc1f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:13.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.289+0000 7f7f01d3a640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc15f100 0x7f7efc0ca680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7eec02a220 tx=0x7f7eec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:13.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.290+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/439897399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec037040 con 0x7f7efc15f100 2024-09-12T01:33:13.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.290+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/439897399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec042070 con 0x7f7efc15f100 2024-09-12T01:33:13.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.290+0000 7f7f0479f640 1 -- 172.21.15.16:0/439897399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7efc1f8100 con 0x7f7efc15f100 2024-09-12T01:33:13.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.290+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/439897399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec030880 con 0x7f7efc15f100 2024-09-12T01:33:13.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.290+0000 7f7f0479f640 1 -- 172.21.15.16:0/439897399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7efc0cabc0 con 0x7f7efc15f100 2024-09-12T01:33:13.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.291+0000 7f7f0479f640 1 -- 172.21.15.16:0/439897399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7efc0d4be0 con 0x7f7efc15f100 2024-09-12T01:33:13.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.291+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/439897399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7eec030a20 con 0x7f7efc15f100 2024-09-12T01:33:13.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.292+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f7ee006be30 0x7f7ee006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:13.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.292+0000 7f7f0253b640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f7ee006be30 0x7f7ee006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:13.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.292+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/439897399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7f7eec0ab650 con 0x7f7efc15f100 2024-09-12T01:33:13.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.292+0000 7f7f0253b640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f7ee006be30 0x7f7ee006e2f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8001b70 tx=0x7f7ef8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:13.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.296+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/439897399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7eec079910 con 0x7f7efc15f100 2024-09-12T01:33:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:13.472+0000 7f7f0479f640 1 -- 172.21.15.16:0/439897399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7efc0b6c60 con 0x7f7efc15f100 2024-09-12T01:33:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.170+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/439897399 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7f7eec07dc10 con 0x7f7efc15f100 2024-09-12T01:33:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' created 2024-09-12T01:33:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.175+0000 7f7f0479f640 1 -- 172.21.15.16:0/439897399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f7ee006be30 msgr2=0x7f7ee006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.175+0000 7f7f0479f640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f7ee006be30 0x7f7ee006e2f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8001b70 tx=0x7f7ef8009040 comp rx=0 tx=0).stop 2024-09-12T01:33:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.175+0000 7f7f0479f640 1 -- 172.21.15.16:0/439897399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc15f100 msgr2=0x7f7efc0ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.175+0000 7f7f0479f640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc15f100 0x7f7efc0ca680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7eec02a220 tx=0x7f7eec0046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.175+0000 7f7f0479f640 1 -- 172.21.15.16:0/439897399 shutdown_connections 2024-09-12T01:33:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.175+0000 7f7f0479f640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f7ee006be30 0x7f7ee006e2f0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8001b70 tx=0x7f7ef8009040 comp rx=0 tx=0).stop 2024-09-12T01:33:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.175+0000 7f7f0479f640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f7efc15f4e0 0x7f7efc15b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.175+0000 7f7f0479f640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc15f100 0x7f7efc0ca680 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7eec02a220 tx=0x7f7eec0046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.175+0000 7f7f0479f640 1 --2- 172.21.15.16:0/439897399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7efc0b6660 0x7f7efc1f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.175+0000 7f7f0479f640 1 -- 172.21.15.16:0/439897399 >> 172.21.15.16:0/439897399 conn(0x7f7efc09d6a0 msgr2=0x7f7efc06d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.176+0000 7f7f0479f640 1 -- 172.21.15.16:0/439897399 shutdown_connections 2024-09-12T01:33:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.176+0000 7f7f0479f640 1 -- 172.21.15.16:0/439897399 wait complete. 2024-09-12T01:33:15.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-12T01:33:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.308+0000 7f87bf186640 1 Processor -- start 2024-09-12T01:33:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.308+0000 7f87bf186640 1 -- start start 2024-09-12T01:33:15.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.309+0000 7f87bf186640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87b8162130 0x7f87b8162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:15.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.309+0000 7f87bf186640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87b80b9a80 0x7f87b80b9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:15.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.309+0000 7f87bf186640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87b80d39d0 con 0x7f87b815e770 2024-09-12T01:33:15.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.309+0000 7f87bf186640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f87b80b7df0 con 0x7f87b8162130 2024-09-12T01:33:15.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.309+0000 7f87bf186640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f87b8162ab0 con 0x7f87b80b9a80 2024-09-12T01:33:15.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.309+0000 7f87bcf22640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f87b815e770 0x7f87b80d32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59036/0 (socket says 172.21.15.16:59036) 2024-09-12T01:33:15.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.309+0000 7f87bcf22640 1 -- 172.21.15.16:0/3678847976 learned_addr learned my addr 172.21.15.16:0/3678847976 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:15.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.309+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3471726125 0 0) 0x7f87b80d39d0 con 0x7f87b815e770 2024-09-12T01:33:15.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.309+0000 7f87bd723640 1 --2- 172.21.15.16:0/3678847976 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87b80b9a80 0x7f87b80b9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:15.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.309+0000 7f87b7fff640 1 --2- 172.21.15.16:0/3678847976 >> v2:172.21.15.33:3300/0 conn(0x7f87b8162130 0x7f87b8162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:15.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.309+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87a8003520 con 0x7f87b815e770 2024-09-12T01:33:15.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.309+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1792189199 0 0) 0x7f87a8003520 con 0x7f87b815e770 2024-09-12T01:33:15.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87b80b9a80 msgr2=0x7f87b80b9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:15.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b77fe640 1 --2- 172.21.15.16:0/3678847976 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87b80b9a80 0x7f87b80b9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:15.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 >> v2:172.21.15.33:3300/0 conn(0x7f87b8162130 msgr2=0x7f87b8162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:15.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b77fe640 1 --2- 172.21.15.16:0/3678847976 >> v2:172.21.15.33:3300/0 conn(0x7f87b8162130 0x7f87b8162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:15.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b80b74c0 con 0x7f87b815e770 2024-09-12T01:33:15.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f87ac001670 con 0x7f87b815e770 2024-09-12T01:33:15.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 >> v1:172.21.15.16:6789/0 conn(0x7f87b815e770 legacy=0x7f87b80d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:15.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b77fe640 1 --2- 172.21.15.16:0/3678847976 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87b80b9a80 0x7f87a80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:15.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b77fe640 1 --2- 172.21.15.16:0/3678847976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a8003f30 0x7f87a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:15.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b77fe640 1 --2- 172.21.15.16:0/3678847976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a8004d40 0x7f87a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:15.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87b80b7df0 con 0x7f87a8004d40 2024-09-12T01:33:15.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8162ab0 con 0x7f87a8003f30 2024-09-12T01:33:15.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f87a800b9c0 con 0x7f87b80b9a80 2024-09-12T01:33:15.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87bd723640 1 --2- 172.21.15.16:0/3678847976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a8004d40 0x7f87a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:15.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b7fff640 1 --2- 172.21.15.16:0/3678847976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a8003f30 0x7f87a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:15.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87bcf22640 1 --2- 172.21.15.16:0/3678847976 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87b80b9a80 0x7f87a80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:15.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87bd723640 1 -- 172.21.15.16:0/3678847976 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87b80b9a80 msgr2=0x7f87a80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:15.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87bd723640 1 --2- 172.21.15.16:0/3678847976 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87b80b9a80 0x7f87a80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:15.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87bd723640 1 -- 172.21.15.16:0/3678847976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a8003f30 msgr2=0x7f87a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:15.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87bd723640 1 --2- 172.21.15.16:0/3678847976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a8003f30 0x7f87a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:15.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87bd723640 1 -- 172.21.15.16:0/3678847976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a8003520 con 0x7f87a8004d40 2024-09-12T01:33:15.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.310+0000 7f87b7fff640 1 --2- 172.21.15.16:0/3678847976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a8003f30 0x7f87a8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:15.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.311+0000 7f87bd723640 1 --2- 172.21.15.16:0/3678847976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a8004d40 0x7f87a80051c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009fc0 tx=0x7f87a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07057a75d567d08 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:15.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.311+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0009c10 con 0x7f87a8004d40 2024-09-12T01:33:15.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.311+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87b0036070 con 0x7f87a8004d40 2024-09-12T01:33:15.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.311+0000 7f87b77fe640 1 -- 172.21.15.16:0/3678847976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0031670 con 0x7f87a8004d40 2024-09-12T01:33:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.311+0000 7f87bf186640 1 -- 172.21.15.16:0/3678847976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a8004d40 msgr2=0x7f87a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.311+0000 7f87bf186640 1 --2- 172.21.15.16:0/3678847976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a8004d40 0x7f87a80051c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009fc0 tx=0x7f87a8005830 comp rx=0 tx=0).stop 2024-09-12T01:33:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.311+0000 7f87bd723640 1 -- 172.21.15.16:0/3678847976 reap_dead start 2024-09-12T01:33:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.311+0000 7f87bf186640 1 -- 172.21.15.16:0/3678847976 shutdown_connections 2024-09-12T01:33:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.311+0000 7f87bf186640 1 -- 172.21.15.16:0/3678847976 >> 172.21.15.16:0/3678847976 conn(0x7f87b809d660 msgr2=0x7f87b80d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.312+0000 7f87bf186640 1 -- 172.21.15.16:0/3678847976 shutdown_connections 2024-09-12T01:33:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.312+0000 7f87bf186640 1 -- 172.21.15.16:0/3678847976 wait complete. 2024-09-12T01:33:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.313+0000 7f87bf186640 1 Processor -- start 2024-09-12T01:33:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.313+0000 7f87bf186640 1 -- start start 2024-09-12T01:33:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bf186640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87a8004d40 0x7f87b8156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bf186640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8161140 0x7f87b815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bf186640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b815c530 0x7f87b80cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bf186640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87b80d51c0 con 0x7f87b815c530 2024-09-12T01:33:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bf186640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87b80b7a80 con 0x7f87b8161140 2024-09-12T01:33:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bf186640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f87b80d5340 con 0x7f87a8004d40 2024-09-12T01:33:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bd723640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b815c530 0x7f87b80cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bd723640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b815c530 0x7f87b80cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58182/0 (socket says 172.21.15.16:58182) 2024-09-12T01:33:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bd723640 1 -- 172.21.15.16:0/3556174182 learned_addr learned my addr 172.21.15.16:0/3556174182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bcf22640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87a8004d40 0x7f87b8156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87b7fff640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8161140 0x7f87b815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bd723640 1 -- 172.21.15.16:0/3556174182 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87a8004d40 msgr2=0x7f87b8156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bd723640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87a8004d40 0x7f87b8156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bd723640 1 -- 172.21.15.16:0/3556174182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8161140 msgr2=0x7f87b815bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bd723640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8161140 0x7f87b815bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bd723640 1 -- 172.21.15.16:0/3556174182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b815fe00 con 0x7f87b815c530 2024-09-12T01:33:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bcf22640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87a8004d40 0x7f87b8156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87b7fff640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8161140 0x7f87b815bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bd723640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b815c530 0x7f87b80cd470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87b000e230 tx=0x7f87b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87b57fa640 1 -- 172.21.15.16:0/3556174182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4009690 con 0x7f87b815c530 2024-09-12T01:33:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87b57fa640 1 -- 172.21.15.16:0/3556174182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87b0031dd0 con 0x7f87b815c530 2024-09-12T01:33:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bf186640 1 -- 172.21.15.16:0/3556174182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b81600a0 con 0x7f87b815c530 2024-09-12T01:33:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87b57fa640 1 -- 172.21.15.16:0/3556174182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0009c10 con 0x7f87b815c530 2024-09-12T01:33:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.314+0000 7f87bf186640 1 -- 172.21.15.16:0/3556174182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b8160660 con 0x7f87b815c530 2024-09-12T01:33:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.316+0000 7f87bf186640 1 -- 172.21.15.16:0/3556174182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b8069110 con 0x7f87b815c530 2024-09-12T01:33:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.316+0000 7f87b57fa640 1 -- 172.21.15.16:0/3556174182 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87b0005d10 con 0x7f87b815c530 2024-09-12T01:33:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.316+0000 7f87b57fa640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f879c06be30 0x7f879c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.317+0000 7f87b57fa640 1 -- 172.21.15.16:0/3556174182 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f87b00ac2a0 con 0x7f87b815c530 2024-09-12T01:33:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.317+0000 7f87bcf22640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f879c06be30 0x7f879c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.317+0000 7f87bcf22640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f879c06be30 0x7f879c06e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f87ac007fd0 tx=0x7f87ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.321+0000 7f87b57fa640 1 -- 172.21.15.16:0/3556174182 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f87b0049060 con 0x7f87b815c530 2024-09-12T01:33:15.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:15.494+0000 7f87bf186640 1 -- 172.21.15.16:0/3556174182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f87b8161520 con 0x7f87b815c530 2024-09-12T01:33:16.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.172+0000 7f87b57fa640 1 -- 172.21.15.16:0/3556174182 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7f87b007a9c0 con 0x7f87b815c530 2024-09-12T01:33:16.172 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 allow_ec_overwrites to true 2024-09-12T01:33:16.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.176+0000 7f87bf186640 1 -- 172.21.15.16:0/3556174182 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f879c06be30 msgr2=0x7f879c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:16.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.176+0000 7f87bf186640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f879c06be30 0x7f879c06e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f87ac007fd0 tx=0x7f87ac009040 comp rx=0 tx=0).stop 2024-09-12T01:33:16.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.176+0000 7f87bf186640 1 -- 172.21.15.16:0/3556174182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b815c530 msgr2=0x7f87b80cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:16.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.176+0000 7f87bf186640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b815c530 0x7f87b80cd470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87b000e230 tx=0x7f87b0009750 comp rx=0 tx=0).stop 2024-09-12T01:33:16.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.177+0000 7f87bf186640 1 -- 172.21.15.16:0/3556174182 shutdown_connections 2024-09-12T01:33:16.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.177+0000 7f87bf186640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f879c06be30 0x7f879c06e2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:16.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.177+0000 7f87bf186640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b815c530 0x7f87b80cd470 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:16.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.177+0000 7f87bf186640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8161140 0x7f87b815bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:16.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.177+0000 7f87bf186640 1 --2- 172.21.15.16:0/3556174182 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f87a8004d40 0x7f87b8156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:16.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.177+0000 7f87bf186640 1 -- 172.21.15.16:0/3556174182 >> 172.21.15.16:0/3556174182 conn(0x7f87b809d660 msgr2=0x7f87b806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:16.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.177+0000 7f87bf186640 1 -- 172.21.15.16:0/3556174182 shutdown_connections 2024-09-12T01:33:16.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.177+0000 7f87bf186640 1 -- 172.21.15.16:0/3556174182 wait complete. 2024-09-12T01:33:16.206 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-12T01:33:16.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.315+0000 7fb567ffd640 1 Processor -- start 2024-09-12T01:33:16.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.315+0000 7fb567ffd640 1 -- start start 2024-09-12T01:33:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.316+0000 7fb567ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb56015b4a0 0x7fb5600ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.316+0000 7fb567ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb56006ebb0 0x7fb56006f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.316+0000 7fb567ffd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb56006f0c0 con 0x7fb56006fda0 2024-09-12T01:33:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.316+0000 7fb567ffd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb56006f3a0 con 0x7fb56015b4a0 2024-09-12T01:33:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.316+0000 7fb567ffd640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb5600cf000 con 0x7fb56006ebb0 2024-09-12T01:33:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.316+0000 7fb565d99640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb56015b4a0 0x7fb5600ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.316+0000 7fb56659a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb56006fda0 0x7fb560072c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58150/0 (socket says 172.21.15.16:58150) 2024-09-12T01:33:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.316+0000 7fb56659a640 1 -- 172.21.15.16:0/1424886872 learned_addr learned my addr 172.21.15.16:0/1424886872 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.316+0000 7fb565598640 1 --2- 172.21.15.16:0/1424886872 >> v2:172.21.15.33:3300/0 conn(0x7fb56006ebb0 0x7fb56006f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.316+0000 7fb565d99640 1 -- 172.21.15.16:0/1424886872 >> v2:172.21.15.33:3300/0 conn(0x7fb56006ebb0 msgr2=0x7fb56006f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.316+0000 7fb565d99640 1 --2- 172.21.15.16:0/1424886872 >> v2:172.21.15.33:3300/0 conn(0x7fb56006ebb0 0x7fb56006f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.316+0000 7fb565d99640 1 -- 172.21.15.16:0/1424886872 >> v1:172.21.15.16:6789/0 conn(0x7fb56006fda0 legacy=0x7fb560072c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.316+0000 7fb565d99640 1 -- 172.21.15.16:0/1424886872 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5600cf2e0 con 0x7fb56015b4a0 2024-09-12T01:33:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.317+0000 7fb565d99640 1 --2- 172.21.15.16:0/1424886872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56015b4a0 0x7fb5600ccfa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0097a0 tx=0x7fb55c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4f7069e17d6b57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.317+0000 7fb564d97640 1 -- 172.21.15.16:0/1424886872 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c037070 con 0x7fb56015b4a0 2024-09-12T01:33:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.317+0000 7fb564d97640 1 -- 172.21.15.16:0/1424886872 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55c032440 con 0x7fb56015b4a0 2024-09-12T01:33:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.317+0000 7fb564d97640 1 -- 172.21.15.16:0/1424886872 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c032810 con 0x7fb56015b4a0 2024-09-12T01:33:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.318+0000 7fb567ffd640 1 -- 172.21.15.16:0/1424886872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56015b4a0 msgr2=0x7fb5600ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.318+0000 7fb567ffd640 1 --2- 172.21.15.16:0/1424886872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56015b4a0 0x7fb5600ccfa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0097a0 tx=0x7fb55c02a1e0 comp rx=0 tx=0).stop 2024-09-12T01:33:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.318+0000 7fb567ffd640 1 -- 172.21.15.16:0/1424886872 shutdown_connections 2024-09-12T01:33:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.318+0000 7fb567ffd640 1 --2- 172.21.15.16:0/1424886872 >> v2:172.21.15.33:3300/0 conn(0x7fb56006ebb0 0x7fb56006f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.318+0000 7fb567ffd640 1 --2- 172.21.15.16:0/1424886872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56015b4a0 0x7fb5600ccfa0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.318+0000 7fb567ffd640 1 -- 172.21.15.16:0/1424886872 >> 172.21.15.16:0/1424886872 conn(0x7fb5600d1100 msgr2=0x7fb5600d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.318+0000 7fb567ffd640 1 -- 172.21.15.16:0/1424886872 shutdown_connections 2024-09-12T01:33:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.318+0000 7fb567ffd640 1 -- 172.21.15.16:0/1424886872 wait complete. 2024-09-12T01:33:16.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.319+0000 7fb567ffd640 1 Processor -- start 2024-09-12T01:33:16.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.319+0000 7fb567ffd640 1 -- start start 2024-09-12T01:33:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb567ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56006ebb0 0x7fb5600ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb567ffd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb56006fda0 0x7fb5600c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb567ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56015b4a0 0x7fb5601526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb567ffd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5600cd660 con 0x7fb56015b4a0 2024-09-12T01:33:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb567ffd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5600cd4e0 con 0x7fb56006ebb0 2024-09-12T01:33:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb567ffd640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb5600d0af0 con 0x7fb56006fda0 2024-09-12T01:33:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb56659a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56015b4a0 0x7fb5601526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb56659a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56015b4a0 0x7fb5601526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59000/0 (socket says 172.21.15.16:59000) 2024-09-12T01:33:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb56659a640 1 -- 172.21.15.16:0/2627883801 learned_addr learned my addr 172.21.15.16:0/2627883801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb565d99640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56006ebb0 0x7fb5600ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb565598640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb56006fda0 0x7fb5600c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb56659a640 1 -- 172.21.15.16:0/2627883801 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb56006fda0 msgr2=0x7fb5600c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb56659a640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb56006fda0 0x7fb5600c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb56659a640 1 -- 172.21.15.16:0/2627883801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56006ebb0 msgr2=0x7fb5600ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb56659a640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56006ebb0 0x7fb5600ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb56659a640 1 -- 172.21.15.16:0/2627883801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5600c5b00 con 0x7fb56015b4a0 2024-09-12T01:33:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb56659a640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56015b4a0 0x7fb5601526b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb55800cbc0 tx=0x7fb55800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb5567fc640 1 -- 172.21.15.16:0/2627883801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558017070 con 0x7fb56015b4a0 2024-09-12T01:33:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb5567fc640 1 -- 172.21.15.16:0/2627883801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55800ad40 con 0x7fb56015b4a0 2024-09-12T01:33:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb5567fc640 1 -- 172.21.15.16:0/2627883801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558012600 con 0x7fb56015b4a0 2024-09-12T01:33:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.320+0000 7fb567ffd640 1 -- 172.21.15.16:0/2627883801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5600c5de0 con 0x7fb56015b4a0 2024-09-12T01:33:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.321+0000 7fb567ffd640 1 -- 172.21.15.16:0/2627883801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5600c63a0 con 0x7fb56015b4a0 2024-09-12T01:33:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.322+0000 7fb567ffd640 1 -- 172.21.15.16:0/2627883801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb56006e2e0 con 0x7fb56015b4a0 2024-09-12T01:33:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.322+0000 7fb5567fc640 1 -- 172.21.15.16:0/2627883801 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb55801a070 con 0x7fb56015b4a0 2024-09-12T01:33:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.323+0000 7fb5567fc640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb54406be30 0x7fb54406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.323+0000 7fb5567fc640 1 -- 172.21.15.16:0/2627883801 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7fb55808ae70 con 0x7fb56015b4a0 2024-09-12T01:33:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.323+0000 7fb565d99640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb54406be30 0x7fb54406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.323+0000 7fb565d99640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb54406be30 0x7fb54406e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb55c034000 tx=0x7fb55c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:16.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.327+0000 7fb5567fc640 1 -- 172.21.15.16:0/2627883801 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb558058eb0 con 0x7fb56015b4a0 2024-09-12T01:33:16.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:16.540+0000 7fb567ffd640 1 -- 172.21.15.16:0/2627883801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fb5600b2280 con 0x7fb56015b4a0 2024-09-12T01:33:17.120 INFO:tasks.ceph.mds.d.smithi033.stderr:2024-09-12T01:33:17.120+0000 7fb6404c9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T01:33:17.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.217+0000 7fb5567fc640 1 -- 172.21.15.16:0/2627883801 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) ==== 137+0+0 (secure 0 0 0) 0x7fb55805d1b0 con 0x7fb56015b4a0 2024-09-12T01:33:17.217 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 4 to fsmap 2024-09-12T01:33:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.220+0000 7fb567ffd640 1 -- 172.21.15.16:0/2627883801 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb54406be30 msgr2=0x7fb54406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.220+0000 7fb567ffd640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb54406be30 0x7fb54406e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb55c034000 tx=0x7fb55c02e000 comp rx=0 tx=0).stop 2024-09-12T01:33:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.220+0000 7fb567ffd640 1 -- 172.21.15.16:0/2627883801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56015b4a0 msgr2=0x7fb5601526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.220+0000 7fb567ffd640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56015b4a0 0x7fb5601526b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb55800cbc0 tx=0x7fb55800a430 comp rx=0 tx=0).stop 2024-09-12T01:33:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.220+0000 7fb567ffd640 1 -- 172.21.15.16:0/2627883801 shutdown_connections 2024-09-12T01:33:17.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.220+0000 7fb567ffd640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb54406be30 0x7fb54406e2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.220+0000 7fb567ffd640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56015b4a0 0x7fb5601526b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.220+0000 7fb567ffd640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb56006fda0 0x7fb5600c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.220+0000 7fb567ffd640 1 --2- 172.21.15.16:0/2627883801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56006ebb0 0x7fb5600ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.220+0000 7fb567ffd640 1 -- 172.21.15.16:0/2627883801 >> 172.21.15.16:0/2627883801 conn(0x7fb5600d1100 msgr2=0x7fb5600b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:17.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.220+0000 7fb567ffd640 1 -- 172.21.15.16:0/2627883801 shutdown_connections 2024-09-12T01:33:17.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.220+0000 7fb567ffd640 1 -- 172.21.15.16:0/2627883801 wait complete. 2024-09-12T01:33:17.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:33:17.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.342+0000 7fc0b6f4b640 1 Processor -- start 2024-09-12T01:33:17.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.343+0000 7fc0b6f4b640 1 -- start start 2024-09-12T01:33:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.343+0000 7fc0b6f4b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc0b0156940 0x7fc0b015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.343+0000 7fc0b6f4b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0b00ccf30 0x7fc0b00cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.343+0000 7fc0b6f4b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0b01608e0 con 0x7fc0b015ff00 2024-09-12T01:33:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.343+0000 7fc0b6f4b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc0b015ce90 con 0x7fc0b00ccf30 2024-09-12T01:33:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.343+0000 7fc0b6f4b640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b0160bc0 con 0x7fc0b0156940 2024-09-12T01:33:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.344+0000 7fc0b54e8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0b00ccf30 0x7fc0b00cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.344+0000 7fc0affff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc0b015ff00 0x7fc0b00cc820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58166/0 (socket says 172.21.15.16:58166) 2024-09-12T01:33:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.344+0000 7fc0b54e8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0b00ccf30 0x7fc0b00cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59004/0 (socket says 172.21.15.16:59004) 2024-09-12T01:33:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.344+0000 7fc0affff640 1 -- 172.21.15.16:0/32928360 learned_addr learned my addr 172.21.15.16:0/32928360 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.344+0000 7fc0b4ce7640 1 --2- 172.21.15.16:0/32928360 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc0b0156940 0x7fc0b015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.344+0000 7fc0b54e8640 1 -- 172.21.15.16:0/32928360 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc0b0156940 msgr2=0x7fc0b015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.344+0000 7fc0b54e8640 1 --2- 172.21.15.16:0/32928360 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc0b0156940 0x7fc0b015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.344+0000 7fc0b54e8640 1 -- 172.21.15.16:0/32928360 >> v1:172.21.15.16:6789/0 conn(0x7fc0b015ff00 legacy=0x7fc0b00cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.344+0000 7fc0b54e8640 1 -- 172.21.15.16:0/32928360 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b015cb20 con 0x7fc0b00ccf30 2024-09-12T01:33:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.345+0000 7fc0b54e8640 1 --2- 172.21.15.16:0/32928360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b00ccf30 0x7fc0b00cfda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009d90 tx=0x7fc0a802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e7229c11889bd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.345+0000 7fc0af7fe640 1 -- 172.21.15.16:0/32928360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8007740 con 0x7fc0b00ccf30 2024-09-12T01:33:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.345+0000 7fc0af7fe640 1 -- 172.21.15.16:0/32928360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a80078e0 con 0x7fc0b00ccf30 2024-09-12T01:33:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.345+0000 7fc0af7fe640 1 -- 172.21.15.16:0/32928360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8007cb0 con 0x7fc0b00ccf30 2024-09-12T01:33:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.345+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/32928360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b00ccf30 msgr2=0x7fc0b00cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.345+0000 7fc0b6f4b640 1 --2- 172.21.15.16:0/32928360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b00ccf30 0x7fc0b00cfda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009d90 tx=0x7fc0a802a660 comp rx=0 tx=0).stop 2024-09-12T01:33:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.345+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/32928360 shutdown_connections 2024-09-12T01:33:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.345+0000 7fc0b6f4b640 1 --2- 172.21.15.16:0/32928360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b00ccf30 0x7fc0b00cfda0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.345+0000 7fc0b6f4b640 1 --2- 172.21.15.16:0/32928360 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc0b0156940 0x7fc0b015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.345+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/32928360 >> 172.21.15.16:0/32928360 conn(0x7fc0b006fdc0 msgr2=0x7fc0b00701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.345+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/32928360 shutdown_connections 2024-09-12T01:33:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.345+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/32928360 wait complete. 2024-09-12T01:33:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.347+0000 7fc0b6f4b640 1 Processor -- start 2024-09-12T01:33:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.347+0000 7fc0b6f4b640 1 -- start start 2024-09-12T01:33:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.347+0000 7fc0b6f4b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0b00ccf30 0x7fc0b0158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.347+0000 7fc0b6f4b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc0b0156940 0x7fc0b0154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.347+0000 7fc0b6f4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b015ff00 0x7fc0b01e5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b6f4b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b00d19b0 con 0x7fc0b015ff00 2024-09-12T01:33:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b6f4b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0156d20 con 0x7fc0b00ccf30 2024-09-12T01:33:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b6f4b640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b00d1b30 con 0x7fc0b0156940 2024-09-12T01:33:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b54e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b015ff00 0x7fc0b01e5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b54e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b015ff00 0x7fc0b01e5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59016/0 (socket says 172.21.15.16:59016) 2024-09-12T01:33:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b54e8640 1 -- 172.21.15.16:0/572747094 learned_addr learned my addr 172.21.15.16:0/572747094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b4ce7640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0b00ccf30 0x7fc0b0158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0affff640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc0b0156940 0x7fc0b0154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b54e8640 1 -- 172.21.15.16:0/572747094 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc0b0156940 msgr2=0x7fc0b0154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b54e8640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc0b0156940 0x7fc0b0154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b54e8640 1 -- 172.21.15.16:0/572747094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0b00ccf30 msgr2=0x7fc0b0158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b54e8640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0b00ccf30 0x7fc0b0158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b54e8640 1 -- 172.21.15.16:0/572747094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b0157b60 con 0x7fc0b015ff00 2024-09-12T01:33:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b54e8640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b015ff00 0x7fc0b01e5bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009eb0 tx=0x7fc0a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0ad7fa640 1 -- 172.21.15.16:0/572747094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8040070 con 0x7fc0b015ff00 2024-09-12T01:33:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0ad7fa640 1 -- 172.21.15.16:0/572747094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a8031bc0 con 0x7fc0b015ff00 2024-09-12T01:33:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/572747094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b0157e40 con 0x7fc0b015ff00 2024-09-12T01:33:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0ad7fa640 1 -- 172.21.15.16:0/572747094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8045460 con 0x7fc0b015ff00 2024-09-12T01:33:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.348+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/572747094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b01586e0 con 0x7fc0b015ff00 2024-09-12T01:33:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.349+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/572747094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b0158230 con 0x7fc0b015ff00 2024-09-12T01:33:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.350+0000 7fc0ad7fa640 1 -- 172.21.15.16:0/572747094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0a8039040 con 0x7fc0b015ff00 2024-09-12T01:33:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.350+0000 7fc0ad7fa640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc09c06be30 0x7fc09c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.350+0000 7fc0ad7fa640 1 -- 172.21.15.16:0/572747094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc0a80ac9d0 con 0x7fc0b015ff00 2024-09-12T01:33:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.351+0000 7fc0b4ce7640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc09c06be30 0x7fc09c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.351+0000 7fc0b4ce7640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc09c06be30 0x7fc09c06e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0006fd0 tx=0x7fc0a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:17.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.355+0000 7fc0ad7fa640 1 -- 172.21.15.16:0/572747094 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc0a807a9f0 con 0x7fc0b015ff00 2024-09-12T01:33:17.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.566+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/572747094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0b0158010 con 0x7fc0b015ff00 2024-09-12T01:33:17.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.567+0000 7fc0ad7fa640 1 -- 172.21.15.16:0/572747094 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5216 (secure 0 0 0) 0x7fc0a807ecf0 con 0x7fc0b015ff00 2024-09-12T01:33:17.567 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:17.567 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"btime":"2024-09-12T01:33:17:212386+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4157213575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4157213575},{"type":"v1","addr":"172.21.15.16:6835","nonce":4157213575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1554674078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1554674078},{"type":"v1","addr":"172.21.15.16:6837","nonce":1554674078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/359404074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":359404074},{"type":"v1","addr":"172.21.15.33:6834","nonce":359404074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:33:11.492995+0000","modified":"2024-09-12T01:33:16.541699+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2179301157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2179301157},{"type":"v1","addr":"172.21.15.33:6835","nonce":2179301157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-12T01:33:17.567 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-09-12T01:33:17.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.569+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/572747094 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc09c06be30 msgr2=0x7fc09c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.569+0000 7fc0b6f4b640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc09c06be30 0x7fc09c06e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0006fd0 tx=0x7fc0a0008040 comp rx=0 tx=0).stop 2024-09-12T01:33:17.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.569+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/572747094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b015ff00 msgr2=0x7fc0b01e5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.569+0000 7fc0b6f4b640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b015ff00 0x7fc0b01e5bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009eb0 tx=0x7fc0a8004a40 comp rx=0 tx=0).stop 2024-09-12T01:33:17.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.569+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/572747094 shutdown_connections 2024-09-12T01:33:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.569+0000 7fc0b6f4b640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc09c06be30 0x7fc09c06e2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.569+0000 7fc0b6f4b640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b015ff00 0x7fc0b01e5bb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.569+0000 7fc0b6f4b640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc0b0156940 0x7fc0b0154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.569+0000 7fc0b6f4b640 1 --2- 172.21.15.16:0/572747094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0b00ccf30 0x7fc0b0158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.569+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/572747094 >> 172.21.15.16:0/572747094 conn(0x7fc0b006fdc0 msgr2=0x7fc0b00cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.569+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/572747094 shutdown_connections 2024-09-12T01:33:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.569+0000 7fc0b6f4b640 1 -- 172.21.15.16:0/572747094 wait complete. 2024-09-12T01:33:17.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:33:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.683+0000 7fa18f0b7640 1 Processor -- start 2024-09-12T01:33:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.683+0000 7fa18f0b7640 1 -- start start 2024-09-12T01:33:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa18f0b7640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa18815e800 0x7fa1880cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa18f0b7640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa1880d3470 0x7fa1880d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa18f0b7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1880cfd30 con 0x7fa1880cf910 2024-09-12T01:33:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa18f0b7640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fa18815ebe0 con 0x7fa18815e800 2024-09-12T01:33:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa18f0b7640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa1880d42d0 con 0x7fa1880d3470 2024-09-12T01:33:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa187fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa1880cf910 0x7fa18806d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58176/0 (socket says 172.21.15.16:58176) 2024-09-12T01:33:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa187fff640 1 -- 172.21.15.16:0/2556142781 learned_addr learned my addr 172.21.15.16:0/2556142781 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa18ce53640 1 --2- 172.21.15.16:0/2556142781 >> v2:172.21.15.33:3300/0 conn(0x7fa18815e800 0x7fa1880cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa18d654640 1 --2- 172.21.15.16:0/2556142781 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa1880d3470 0x7fa1880d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2520243610 0 0) 0x7fa1880cfd30 con 0x7fa1880cf910 2024-09-12T01:33:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa178003520 con 0x7fa1880cf910 2024-09-12T01:33:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1777760935 0 0) 0x7fa178003520 con 0x7fa1880cf910 2024-09-12T01:33:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa1880d3470 msgr2=0x7fa1880d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa1877fe640 1 --2- 172.21.15.16:0/2556142781 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa1880d3470 0x7fa1880d3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 >> v2:172.21.15.33:3300/0 conn(0x7fa18815e800 msgr2=0x7fa1880cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa1877fe640 1 --2- 172.21.15.16:0/2556142781 >> v2:172.21.15.33:3300/0 conn(0x7fa18815e800 0x7fa1880cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1880d3ff0 con 0x7fa1880cf910 2024-09-12T01:33:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7fa174001670 con 0x7fa1880cf910 2024-09-12T01:33:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.684+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 >> v1:172.21.15.16:6789/0 conn(0x7fa1880cf910 legacy=0x7fa18806d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa1877fe640 1 --2- 172.21.15.16:0/2556142781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178003a20 0x7fa178003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa1877fe640 1 --2- 172.21.15.16:0/2556142781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa178004800 0x7fa178004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa1877fe640 1 --2- 172.21.15.16:0/2556142781 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa1880d3470 0x7fa1780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa18815ebe0 con 0x7fa178003a20 2024-09-12T01:33:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1880d42d0 con 0x7fa178004800 2024-09-12T01:33:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa17800b9c0 con 0x7fa1880d3470 2024-09-12T01:33:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa18ce53640 1 --2- 172.21.15.16:0/2556142781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178003a20 0x7fa178003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa187fff640 1 --2- 172.21.15.16:0/2556142781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa178004800 0x7fa178004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa18d654640 1 --2- 172.21.15.16:0/2556142781 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa1880d3470 0x7fa1780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa18ce53640 1 -- 172.21.15.16:0/2556142781 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa1880d3470 msgr2=0x7fa1780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa18ce53640 1 --2- 172.21.15.16:0/2556142781 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa1880d3470 0x7fa1780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa18ce53640 1 -- 172.21.15.16:0/2556142781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa178004800 msgr2=0x7fa178004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa18ce53640 1 --2- 172.21.15.16:0/2556142781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa178004800 0x7fa178004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa18ce53640 1 -- 172.21.15.16:0/2556142781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa178003520 con 0x7fa178003a20 2024-09-12T01:33:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.685+0000 7fa187fff640 1 --2- 172.21.15.16:0/2556142781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa178004800 0x7fa178004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.686+0000 7fa18ce53640 1 --2- 172.21.15.16:0/2556142781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178003a20 0x7fa178003e40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009af0 tx=0x7fa178005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9099ace7efe912ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.686+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c036070 con 0x7fa178003a20 2024-09-12T01:33:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.686+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa17c031400 con 0x7fa178003a20 2024-09-12T01:33:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.686+0000 7fa1877fe640 1 -- 172.21.15.16:0/2556142781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c009690 con 0x7fa178003a20 2024-09-12T01:33:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.686+0000 7fa18f0b7640 1 -- 172.21.15.16:0/2556142781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178003a20 msgr2=0x7fa178003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.686+0000 7fa18f0b7640 1 --2- 172.21.15.16:0/2556142781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178003a20 0x7fa178003e40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009af0 tx=0x7fa178005830 comp rx=0 tx=0).stop 2024-09-12T01:33:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.686+0000 7fa18d654640 1 -- 172.21.15.16:0/2556142781 reap_dead start 2024-09-12T01:33:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.686+0000 7fa18f0b7640 1 -- 172.21.15.16:0/2556142781 shutdown_connections 2024-09-12T01:33:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.686+0000 7fa18f0b7640 1 -- 172.21.15.16:0/2556142781 >> 172.21.15.16:0/2556142781 conn(0x7fa18809d750 msgr2=0x7fa1880d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.686+0000 7fa18f0b7640 1 -- 172.21.15.16:0/2556142781 shutdown_connections 2024-09-12T01:33:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.686+0000 7fa18f0b7640 1 -- 172.21.15.16:0/2556142781 wait complete. 2024-09-12T01:33:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.688+0000 7fa18f0b7640 1 Processor -- start 2024-09-12T01:33:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.688+0000 7fa18f0b7640 1 -- start start 2024-09-12T01:33:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.688+0000 7fa18f0b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa178003a20 0x7fa18814e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.688+0000 7fa18f0b7640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa1880b8cd0 0x7fa1880b9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.688+0000 7fa18f0b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1880cb5d0 0x7fa1880cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.688+0000 7fa18f0b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa18806d170 con 0x7fa1880cb5d0 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.688+0000 7fa18f0b7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa18806ce90 con 0x7fa178003a20 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.688+0000 7fa18f0b7640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa1880d4e30 con 0x7fa1880b8cd0 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa18d654640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1880cb5d0 0x7fa1880cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa18d654640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1880cb5d0 0x7fa1880cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59020/0 (socket says 172.21.15.16:59020) 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa18d654640 1 -- 172.21.15.16:0/1297259402 learned_addr learned my addr 172.21.15.16:0/1297259402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa18ce53640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa178003a20 0x7fa18814e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa187fff640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa1880b8cd0 0x7fa1880b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa18d654640 1 -- 172.21.15.16:0/1297259402 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa1880b8cd0 msgr2=0x7fa1880b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa18d654640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa1880b8cd0 0x7fa1880b9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa18d654640 1 -- 172.21.15.16:0/1297259402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa178003a20 msgr2=0x7fa18814e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa18d654640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa178003a20 0x7fa18814e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa18d654640 1 -- 172.21.15.16:0/1297259402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18814eef0 con 0x7fa1880cb5d0 2024-09-12T01:33:17.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa18ce53640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa178003a20 0x7fa18814e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:17.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa18d654640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1880cb5d0 0x7fa1880cba50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa18808cad0 tx=0x7fa1800115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:17.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa1857fa640 1 -- 172.21.15.16:0/1297259402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180011db0 con 0x7fa1880cb5d0 2024-09-12T01:33:17.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa1857fa640 1 -- 172.21.15.16:0/1297259402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa18001b070 con 0x7fa1880cb5d0 2024-09-12T01:33:17.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa18f0b7640 1 -- 172.21.15.16:0/1297259402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1880b9780 con 0x7fa1880cb5d0 2024-09-12T01:33:17.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa1857fa640 1 -- 172.21.15.16:0/1297259402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180003990 con 0x7fa1880cb5d0 2024-09-12T01:33:17.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.689+0000 7fa18f0b7640 1 -- 172.21.15.16:0/1297259402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa18815fb10 con 0x7fa1880cb5d0 2024-09-12T01:33:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.690+0000 7fa18f0b7640 1 -- 172.21.15.16:0/1297259402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa188068880 con 0x7fa1880cb5d0 2024-09-12T01:33:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.691+0000 7fa1857fa640 1 -- 172.21.15.16:0/1297259402 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa180005d10 con 0x7fa1880cb5d0 2024-09-12T01:33:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.691+0000 7fa1857fa640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa16c06be60 0x7fa16c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.691+0000 7fa1857fa640 1 -- 172.21.15.16:0/1297259402 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa18008b650 con 0x7fa1880cb5d0 2024-09-12T01:33:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.691+0000 7fa18ce53640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa16c06be60 0x7fa16c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.692+0000 7fa18ce53640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa16c06be60 0x7fa16c06e320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa17c033000 tx=0x7fa17c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.696+0000 7fa1857fa640 1 -- 172.21.15.16:0/1297259402 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa180025020 con 0x7fa1880cb5d0 2024-09-12T01:33:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.872+0000 7fa18f0b7640 1 -- 172.21.15.16:0/1297259402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1880b9ca0 con 0x7fa1880cb5d0 2024-09-12T01:33:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.873+0000 7fa1857fa640 1 -- 172.21.15.16:0/1297259402 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18587 (secure 0 0 0) 0x7fa1880b9ca0 con 0x7fa1880cb5d0 2024-09-12T01:33:17.874 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:17.874 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:33:16.541441+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:33:09.364639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:33:10.478237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:33:14.204624+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:33:17.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.875+0000 7fa18f0b7640 1 -- 172.21.15.16:0/1297259402 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa16c06be60 msgr2=0x7fa16c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.875+0000 7fa18f0b7640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa16c06be60 0x7fa16c06e320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa17c033000 tx=0x7fa17c008030 comp rx=0 tx=0).stop 2024-09-12T01:33:17.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.875+0000 7fa18f0b7640 1 -- 172.21.15.16:0/1297259402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1880cb5d0 msgr2=0x7fa1880cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.875+0000 7fa18f0b7640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1880cb5d0 0x7fa1880cba50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa18808cad0 tx=0x7fa1800115a0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.875+0000 7fa18f0b7640 1 -- 172.21.15.16:0/1297259402 shutdown_connections 2024-09-12T01:33:17.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.875+0000 7fa18f0b7640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa16c06be60 0x7fa16c06e320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.875+0000 7fa18f0b7640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1880cb5d0 0x7fa1880cba50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.875+0000 7fa18f0b7640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa1880b8cd0 0x7fa1880b9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.875+0000 7fa18f0b7640 1 --2- 172.21.15.16:0/1297259402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa178003a20 0x7fa18814e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.876+0000 7fa18f0b7640 1 -- 172.21.15.16:0/1297259402 >> 172.21.15.16:0/1297259402 conn(0x7fa18809d750 msgr2=0x7fa1880b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:17.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.876+0000 7fa18f0b7640 1 -- 172.21.15.16:0/1297259402 shutdown_connections 2024-09-12T01:33:17.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.876+0000 7fa18f0b7640 1 -- 172.21.15.16:0/1297259402 wait complete. 2024-09-12T01:33:17.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:33:17.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.991+0000 7f551651f640 1 Processor -- start 2024-09-12T01:33:17.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.991+0000 7f551651f640 1 -- start start 2024-09-12T01:33:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.992+0000 7f551651f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f551015e510 0x7f55100ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.992+0000 7f551651f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55100d13d0 0x7f55100b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.992+0000 7f551651f640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55100d3430 con 0x7f55100cf040 2024-09-12T01:33:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.992+0000 7f551651f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f55100d36d0 con 0x7f551015e510 2024-09-12T01:33:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.992+0000 7f551651f640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f55100d39b0 con 0x7f55100d13d0 2024-09-12T01:33:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.992+0000 7f550ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f551015e510 0x7f55100ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.992+0000 7f550ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f551015e510 0x7f55100ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59022/0 (socket says 172.21.15.16:59022) 2024-09-12T01:33:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.992+0000 7f550ffff640 1 -- 172.21.15.16:0/1788473261 learned_addr learned my addr 172.21.15.16:0/1788473261 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.993+0000 7f550ffff640 1 -- 172.21.15.16:0/1788473261 >> v2:172.21.15.33:3300/0 conn(0x7f55100d13d0 msgr2=0x7f55100b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:17.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.993+0000 7f550ffff640 1 --2- 172.21.15.16:0/1788473261 >> v2:172.21.15.33:3300/0 conn(0x7f55100d13d0 0x7f55100b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.993+0000 7f550ffff640 1 -- 172.21.15.16:0/1788473261 >> v1:172.21.15.33:6789/0 conn(0x7f55100cf040 legacy=0x7f55100d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:17.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.993+0000 7f550ffff640 1 -- 172.21.15.16:0/1788473261 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55100d3c90 con 0x7f551015e510 2024-09-12T01:33:17.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.993+0000 7f550ffff640 1 --2- 172.21.15.16:0/1788473261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f551015e510 0x7f55100ceb00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f55100b9dc0 tx=0x7f550002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff0f73a4a2d387c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:17.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.993+0000 7f550effd640 1 -- 172.21.15.16:0/1788473261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500037070 con 0x7f551015e510 2024-09-12T01:33:17.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.993+0000 7f550effd640 1 -- 172.21.15.16:0/1788473261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5500032440 con 0x7f551015e510 2024-09-12T01:33:17.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.993+0000 7f550effd640 1 -- 172.21.15.16:0/1788473261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500032810 con 0x7f551015e510 2024-09-12T01:33:17.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.994+0000 7f551651f640 1 -- 172.21.15.16:0/1788473261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f551015e510 msgr2=0x7f55100ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.994+0000 7f551651f640 1 --2- 172.21.15.16:0/1788473261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f551015e510 0x7f55100ceb00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f55100b9dc0 tx=0x7f550002a1a0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.994+0000 7f551651f640 1 -- 172.21.15.16:0/1788473261 shutdown_connections 2024-09-12T01:33:17.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.994+0000 7f551651f640 1 --2- 172.21.15.16:0/1788473261 >> v2:172.21.15.33:3300/0 conn(0x7f55100d13d0 0x7f55100b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.994+0000 7f551651f640 1 --2- 172.21.15.16:0/1788473261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f551015e510 0x7f55100ceb00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.994+0000 7f551651f640 1 -- 172.21.15.16:0/1788473261 >> 172.21.15.16:0/1788473261 conn(0x7f551006d1a0 msgr2=0x7f5510073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:17.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.994+0000 7f551651f640 1 -- 172.21.15.16:0/1788473261 shutdown_connections 2024-09-12T01:33:17.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.994+0000 7f551651f640 1 -- 172.21.15.16:0/1788473261 wait complete. 2024-09-12T01:33:17.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.995+0000 7f551651f640 1 Processor -- start 2024-09-12T01:33:17.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.995+0000 7f551651f640 1 -- start start 2024-09-12T01:33:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f551651f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55100cf040 0x7f55101514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f551651f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55100d13d0 0x7f5510160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f551651f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f551015e510 0x7f5510161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f551651f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f551015e8f0 con 0x7f55100d13d0 2024-09-12T01:33:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f551651f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55100d22a0 con 0x7f55100cf040 2024-09-12T01:33:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f551651f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f551015ea70 con 0x7f551015e510 2024-09-12T01:33:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f550f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55100d13d0 0x7f5510160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f550f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55100d13d0 0x7f5510160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59024/0 (socket says 172.21.15.16:59024) 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f550f7fe640 1 -- 172.21.15.16:0/2750984371 learned_addr learned my addr 172.21.15.16:0/2750984371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f550ffff640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55100cf040 0x7f55101514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f5514abc640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f551015e510 0x7f5510161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f550f7fe640 1 -- 172.21.15.16:0/2750984371 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f551015e510 msgr2=0x7f5510161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f550f7fe640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f551015e510 0x7f5510161020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f550f7fe640 1 -- 172.21.15.16:0/2750984371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55100cf040 msgr2=0x7f55101514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.996+0000 7f550f7fe640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55100cf040 0x7f55101514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.997+0000 7f550f7fe640 1 -- 172.21.15.16:0/2750984371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5510161b80 con 0x7f55100d13d0 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.997+0000 7f550f7fe640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55100d13d0 0x7f5510160ae0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f550400c410 tx=0x7f550400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.997+0000 7f550cff9640 1 -- 172.21.15.16:0/2750984371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504017070 con 0x7f55100d13d0 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.997+0000 7f550cff9640 1 -- 172.21.15.16:0/2750984371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55040035f0 con 0x7f55100d13d0 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.997+0000 7f551651f640 1 -- 172.21.15.16:0/2750984371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5510151a10 con 0x7f55100d13d0 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.997+0000 7f550cff9640 1 -- 172.21.15.16:0/2750984371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504003a20 con 0x7f55100d13d0 2024-09-12T01:33:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.997+0000 7f551651f640 1 -- 172.21.15.16:0/2750984371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f551015f9a0 con 0x7f55100d13d0 2024-09-12T01:33:18.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.998+0000 7f551651f640 1 -- 172.21.15.16:0/2750984371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55100ceb00 con 0x7f55100d13d0 2024-09-12T01:33:18.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.998+0000 7f550cff9640 1 -- 172.21.15.16:0/2750984371 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f550401a070 con 0x7f55100d13d0 2024-09-12T01:33:18.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.999+0000 7f550cff9640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f54fc06be30 0x7f54fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:18.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.999+0000 7f550cff9640 1 -- 172.21.15.16:0/2750984371 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f550408a510 con 0x7f55100d13d0 2024-09-12T01:33:18.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.999+0000 7f550ffff640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f54fc06be30 0x7f54fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:18.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:17.999+0000 7f550ffff640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f54fc06be30 0x7f54fc06e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5500034000 tx=0x7f550002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:18.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.003+0000 7f550cff9640 1 -- 172.21.15.16:0/2750984371 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f55040010c0 con 0x7f55100d13d0 2024-09-12T01:33:18.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.181+0000 7f551651f640 1 -- 172.21.15.16:0/2750984371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f551015bc20 con 0x7f55100d13d0 2024-09-12T01:33:18.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.182+0000 7f550cff9640 1 -- 172.21.15.16:0/2750984371 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18587 (secure 0 0 0) 0x7f55040585c0 con 0x7f55100d13d0 2024-09-12T01:33:18.182 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:33:16.541441+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:33:09.364639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:33:10.478237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:33:14.204624+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:33:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.185+0000 7f551651f640 1 -- 172.21.15.16:0/2750984371 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f54fc06be30 msgr2=0x7f54fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.185+0000 7f551651f640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f54fc06be30 0x7f54fc06e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5500034000 tx=0x7f550002e000 comp rx=0 tx=0).stop 2024-09-12T01:33:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.185+0000 7f551651f640 1 -- 172.21.15.16:0/2750984371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55100d13d0 msgr2=0x7f5510160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.185+0000 7f551651f640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55100d13d0 0x7f5510160ae0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f550400c410 tx=0x7f550400c8d0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.185+0000 7f551651f640 1 -- 172.21.15.16:0/2750984371 shutdown_connections 2024-09-12T01:33:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.185+0000 7f551651f640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f54fc06be30 0x7f54fc06e2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.185+0000 7f551651f640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f551015e510 0x7f5510161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.185+0000 7f551651f640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55100d13d0 0x7f5510160ae0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.185+0000 7f551651f640 1 --2- 172.21.15.16:0/2750984371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55100cf040 0x7f55101514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.185+0000 7f551651f640 1 -- 172.21.15.16:0/2750984371 >> 172.21.15.16:0/2750984371 conn(0x7f551006d1a0 msgr2=0x7f55100b96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.185+0000 7f551651f640 1 -- 172.21.15.16:0/2750984371 shutdown_connections 2024-09-12T01:33:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.185+0000 7f551651f640 1 -- 172.21.15.16:0/2750984371 wait complete. 2024-09-12T01:33:18.213 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:33:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.316+0000 7fdd40a1e640 1 Processor -- start 2024-09-12T01:33:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.316+0000 7fdd40a1e640 1 -- start start 2024-09-12T01:33:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd40a1e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdd3c15e510 0x7fdd3c0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd40a1e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd3c0cf040 0x7fdd3c0d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd40a1e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd3c0d3430 con 0x7fdd3c0d1200 2024-09-12T01:33:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd40a1e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fdd3c0d1670 con 0x7fdd3c0cf040 2024-09-12T01:33:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd40a1e640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fdd3c0d36d0 con 0x7fdd3c15e510 2024-09-12T01:33:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd39d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd3c0cf040 0x7fdd3c0d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd3ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdd3c0d1200 0x7fdd3c0b5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58192/0 (socket says 172.21.15.16:58192) 2024-09-12T01:33:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd3ad9d640 1 -- 172.21.15.16:0/417158308 learned_addr learned my addr 172.21.15.16:0/417158308 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd3a59c640 1 --2- 172.21.15.16:0/417158308 >> v2:172.21.15.33:3300/0 conn(0x7fdd3c15e510 0x7fdd3c0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd39d9b640 1 -- 172.21.15.16:0/417158308 >> v2:172.21.15.33:3300/0 conn(0x7fdd3c15e510 msgr2=0x7fdd3c0ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd39d9b640 1 --2- 172.21.15.16:0/417158308 >> v2:172.21.15.33:3300/0 conn(0x7fdd3c15e510 0x7fdd3c0ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd39d9b640 1 -- 172.21.15.16:0/417158308 >> v1:172.21.15.16:6789/0 conn(0x7fdd3c0d1200 legacy=0x7fdd3c0b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd3959a640 1 -- 172.21.15.16:0/417158308 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 533611093 0 0) 0x7fdd3c0d3430 con 0x7fdd3c0d1200 2024-09-12T01:33:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd39d9b640 1 -- 172.21.15.16:0/417158308 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c0d39b0 con 0x7fdd3c0cf040 2024-09-12T01:33:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.317+0000 7fdd3a59c640 1 --2- 172.21.15.16:0/417158308 >> v2:172.21.15.33:3300/0 conn(0x7fdd3c15e510 0x7fdd3c0ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.318+0000 7fdd39d9b640 1 --2- 172.21.15.16:0/417158308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd3c0cf040 0x7fdd3c0d0cc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0cb530 tx=0x7fdd3002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ad4dfd7303a8da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.318+0000 7fdd3959a640 1 -- 172.21.15.16:0/417158308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30037070 con 0x7fdd3c0cf040 2024-09-12T01:33:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.318+0000 7fdd3959a640 1 -- 172.21.15.16:0/417158308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd30032440 con 0x7fdd3c0cf040 2024-09-12T01:33:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.318+0000 7fdd3959a640 1 -- 172.21.15.16:0/417158308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3002ae60 con 0x7fdd3c0cf040 2024-09-12T01:33:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.319+0000 7fdd40a1e640 1 -- 172.21.15.16:0/417158308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd3c0cf040 msgr2=0x7fdd3c0d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.319+0000 7fdd40a1e640 1 --2- 172.21.15.16:0/417158308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd3c0cf040 0x7fdd3c0d0cc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0cb530 tx=0x7fdd3002a110 comp rx=0 tx=0).stop 2024-09-12T01:33:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.319+0000 7fdd40a1e640 1 -- 172.21.15.16:0/417158308 shutdown_connections 2024-09-12T01:33:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.319+0000 7fdd40a1e640 1 --2- 172.21.15.16:0/417158308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd3c0cf040 0x7fdd3c0d0cc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.319+0000 7fdd40a1e640 1 --2- 172.21.15.16:0/417158308 >> v2:172.21.15.33:3300/0 conn(0x7fdd3c15e510 0x7fdd3c0ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.319+0000 7fdd40a1e640 1 -- 172.21.15.16:0/417158308 >> 172.21.15.16:0/417158308 conn(0x7fdd3c06d1a0 msgr2=0x7fdd3c073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.319+0000 7fdd40a1e640 1 -- 172.21.15.16:0/417158308 shutdown_connections 2024-09-12T01:33:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.319+0000 7fdd40a1e640 1 -- 172.21.15.16:0/417158308 wait complete. 2024-09-12T01:33:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.320+0000 7fdd40a1e640 1 Processor -- start 2024-09-12T01:33:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.320+0000 7fdd40a1e640 1 -- start start 2024-09-12T01:33:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd40a1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd3c0cf040 0x7fdd3c0ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd40a1e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd3c0d1200 0x7fdd3c14ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd40a1e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fdd3c15e510 0x7fdd3c145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd40a1e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c0d22a0 con 0x7fdd3c0cf040 2024-09-12T01:33:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd40a1e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c0d3f70 con 0x7fdd3c0d1200 2024-09-12T01:33:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd40a1e640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3c15e8f0 con 0x7fdd3c15e510 2024-09-12T01:33:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd3a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd3c0cf040 0x7fdd3c0ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd3a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd3c0cf040 0x7fdd3c0ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59042/0 (socket says 172.21.15.16:59042) 2024-09-12T01:33:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd3a59c640 1 -- 172.21.15.16:0/2457151643 learned_addr learned my addr 172.21.15.16:0/2457151643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd39d9b640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd3c0d1200 0x7fdd3c14ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd3ad9d640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fdd3c15e510 0x7fdd3c145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd3a59c640 1 -- 172.21.15.16:0/2457151643 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fdd3c15e510 msgr2=0x7fdd3c145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd3a59c640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fdd3c15e510 0x7fdd3c145180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd3a59c640 1 -- 172.21.15.16:0/2457151643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd3c0d1200 msgr2=0x7fdd3c14ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd3a59c640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd3c0d1200 0x7fdd3c14ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd3a59c640 1 -- 172.21.15.16:0/2457151643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c146030 con 0x7fdd3c0cf040 2024-09-12T01:33:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.321+0000 7fdd3ad9d640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fdd3c15e510 0x7fdd3c145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.322+0000 7fdd3a59c640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd3c0cf040 0x7fdd3c0ce110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00c7f0 tx=0x7fdd2c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.322+0000 7fdd2affd640 1 -- 172.21.15.16:0/2457151643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c015070 con 0x7fdd3c0cf040 2024-09-12T01:33:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.322+0000 7fdd2affd640 1 -- 172.21.15.16:0/2457151643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd2c007a30 con 0x7fdd3c0cf040 2024-09-12T01:33:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.322+0000 7fdd40a1e640 1 -- 172.21.15.16:0/2457151643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3c161830 con 0x7fdd3c0cf040 2024-09-12T01:33:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.322+0000 7fdd2affd640 1 -- 172.21.15.16:0/2457151643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c007e00 con 0x7fdd3c0cf040 2024-09-12T01:33:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.322+0000 7fdd40a1e640 1 -- 172.21.15.16:0/2457151643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3c161b10 con 0x7fdd3c0cf040 2024-09-12T01:33:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.323+0000 7fdd40a1e640 1 -- 172.21.15.16:0/2457151643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3c0d0cc0 con 0x7fdd3c0cf040 2024-09-12T01:33:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.323+0000 7fdd2affd640 1 -- 172.21.15.16:0/2457151643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd2c018070 con 0x7fdd3c0cf040 2024-09-12T01:33:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.324+0000 7fdd2affd640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fdd1c06be30 0x7fdd1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.324+0000 7fdd2affd640 1 -- 172.21.15.16:0/2457151643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdd2c08b070 con 0x7fdd3c0cf040 2024-09-12T01:33:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.324+0000 7fdd39d9b640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fdd1c06be30 0x7fdd1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.324+0000 7fdd39d9b640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fdd1c06be30 0x7fdd1c06e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdd30034000 tx=0x7fdd3002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.328+0000 7fdd2affd640 1 -- 172.21.15.16:0/2457151643 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdd2c059090 con 0x7fdd3c0cf040 2024-09-12T01:33:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.537+0000 7fdd40a1e640 1 -- 172.21.15.16:0/2457151643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd3c0cd840 con 0x7fdd3c0cf040 2024-09-12T01:33:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.537+0000 7fdd2affd640 1 -- 172.21.15.16:0/2457151643 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5216 (secure 0 0 0) 0x7fdd3c0cd840 con 0x7fdd3c0cf040 2024-09-12T01:33:18.538 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:18.538 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"btime":"2024-09-12T01:33:17:212386+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4157213575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4157213575},{"type":"v1","addr":"172.21.15.16:6835","nonce":4157213575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1554674078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1554674078},{"type":"v1","addr":"172.21.15.16:6837","nonce":1554674078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/359404074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":359404074},{"type":"v1","addr":"172.21.15.33:6834","nonce":359404074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:33:11.492995+0000","modified":"2024-09-12T01:33:16.541699+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2179301157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2179301157},{"type":"v1","addr":"172.21.15.33:6835","nonce":2179301157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-12T01:33:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-09-12T01:33:18.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.539+0000 7fdd40a1e640 1 -- 172.21.15.16:0/2457151643 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fdd1c06be30 msgr2=0x7fdd1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.539+0000 7fdd40a1e640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fdd1c06be30 0x7fdd1c06e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdd30034000 tx=0x7fdd3002e000 comp rx=0 tx=0).stop 2024-09-12T01:33:18.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.539+0000 7fdd40a1e640 1 -- 172.21.15.16:0/2457151643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd3c0cf040 msgr2=0x7fdd3c0ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.539+0000 7fdd40a1e640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd3c0cf040 0x7fdd3c0ce110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00c7f0 tx=0x7fdd2c00cbf0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.539+0000 7fdd40a1e640 1 -- 172.21.15.16:0/2457151643 shutdown_connections 2024-09-12T01:33:18.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.539+0000 7fdd40a1e640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fdd1c06be30 0x7fdd1c06e2f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.539+0000 7fdd40a1e640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fdd3c15e510 0x7fdd3c145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.539+0000 7fdd40a1e640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd3c0d1200 0x7fdd3c14ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.539+0000 7fdd40a1e640 1 --2- 172.21.15.16:0/2457151643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd3c0cf040 0x7fdd3c0ce110 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.540+0000 7fdd40a1e640 1 -- 172.21.15.16:0/2457151643 >> 172.21.15.16:0/2457151643 conn(0x7fdd3c06d1a0 msgr2=0x7fdd3c159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:18.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.540+0000 7fdd40a1e640 1 -- 172.21.15.16:0/2457151643 shutdown_connections 2024-09-12T01:33:18.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.540+0000 7fdd40a1e640 1 -- 172.21.15.16:0/2457151643 wait complete. 2024-09-12T01:33:18.550 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:33:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.652+0000 7f6ea4bd8640 1 Processor -- start 2024-09-12T01:33:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.652+0000 7f6ea4bd8640 1 -- start start 2024-09-12T01:33:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.653+0000 7f6ea4bd8640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ea006baf0 0x7f6ea006bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.653+0000 7f6ea4bd8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6ea00b47f0 0x7f6ea00b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.653+0000 7f6ea4bd8640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ea0073ec0 con 0x7f6ea015e760 2024-09-12T01:33:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.653+0000 7f6ea4bd8640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f6ea00cf6f0 con 0x7f6ea006baf0 2024-09-12T01:33:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.653+0000 7f6ea4bd8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea006c410 con 0x7f6ea00b47f0 2024-09-12T01:33:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.654+0000 7f6e9ed9d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6ea00b47f0 0x7f6ea00b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.654+0000 7f6e9e59c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ea006baf0 0x7f6ea006bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.654+0000 7f6e9dd9b640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6ea015e760 0x7f6ea00cefe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:34774/0 (socket says 172.21.15.16:34774) 2024-09-12T01:33:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.654+0000 7f6e9dd9b640 1 -- 172.21.15.16:0/1927997359 learned_addr learned my addr 172.21.15.16:0/1927997359 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.654+0000 7f6e9ed9d640 1 -- 172.21.15.16:0/1927997359 >> v2:172.21.15.33:3300/0 conn(0x7f6ea006baf0 msgr2=0x7f6ea006bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.654+0000 7f6e9ed9d640 1 --2- 172.21.15.16:0/1927997359 >> v2:172.21.15.33:3300/0 conn(0x7f6ea006baf0 0x7f6ea006bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.654+0000 7f6e9ed9d640 1 -- 172.21.15.16:0/1927997359 >> v1:172.21.15.33:6789/0 conn(0x7f6ea015e760 legacy=0x7f6ea00cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.654+0000 7f6e9ed9d640 1 -- 172.21.15.16:0/1927997359 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea0072e20 con 0x7f6ea00b47f0 2024-09-12T01:33:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.655+0000 7f6e9ed9d640 1 --2- 172.21.15.16:0/1927997359 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6ea00b47f0 0x7f6ea00b4c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009d90 tx=0x7f6e9802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=748116ad3bed1667 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.655+0000 7f6e9d59a640 1 -- 172.21.15.16:0/1927997359 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98031600 con 0x7f6ea00b47f0 2024-09-12T01:33:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.655+0000 7f6e9d59a640 1 -- 172.21.15.16:0/1927997359 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e980317a0 con 0x7f6ea00b47f0 2024-09-12T01:33:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.655+0000 7f6e9d59a640 1 -- 172.21.15.16:0/1927997359 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98031b70 con 0x7f6ea00b47f0 2024-09-12T01:33:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.656+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/1927997359 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6ea00b47f0 msgr2=0x7f6ea00b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.656+0000 7f6ea4bd8640 1 --2- 172.21.15.16:0/1927997359 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6ea00b47f0 0x7f6ea00b4c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009d90 tx=0x7f6e9802a5d0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.656+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/1927997359 shutdown_connections 2024-09-12T01:33:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.656+0000 7f6ea4bd8640 1 --2- 172.21.15.16:0/1927997359 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6ea00b47f0 0x7f6ea00b4c80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.656+0000 7f6ea4bd8640 1 --2- 172.21.15.16:0/1927997359 >> v2:172.21.15.33:3300/0 conn(0x7f6ea006baf0 0x7f6ea006bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.656+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/1927997359 >> 172.21.15.16:0/1927997359 conn(0x7f6ea00b2c50 msgr2=0x7f6ea00b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.656+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/1927997359 shutdown_connections 2024-09-12T01:33:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.656+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/1927997359 wait complete. 2024-09-12T01:33:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.657+0000 7f6ea4bd8640 1 Processor -- start 2024-09-12T01:33:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.657+0000 7f6ea4bd8640 1 -- start start 2024-09-12T01:33:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6ea4bd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ea006baf0 0x7f6ea0150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6ea4bd8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea00b47f0 0x7f6ea0156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6ea4bd8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6ea015e760 0x7f6ea006f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6ea4bd8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea00736c0 con 0x7f6ea006baf0 2024-09-12T01:33:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6ea4bd8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea00733e0 con 0x7f6ea00b47f0 2024-09-12T01:33:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6ea4bd8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea00739a0 con 0x7f6ea015e760 2024-09-12T01:33:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6e9e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ea006baf0 0x7f6ea0150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6e9e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ea006baf0 0x7f6ea0150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59046/0 (socket says 172.21.15.16:59046) 2024-09-12T01:33:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6e9e59c640 1 -- 172.21.15.16:0/653454176 learned_addr learned my addr 172.21.15.16:0/653454176 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6e9dd9b640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea00b47f0 0x7f6ea0156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6e9ed9d640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6ea015e760 0x7f6ea006f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6e9e59c640 1 -- 172.21.15.16:0/653454176 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6ea015e760 msgr2=0x7f6ea006f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6e9e59c640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6ea015e760 0x7f6ea006f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6e9e59c640 1 -- 172.21.15.16:0/653454176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea00b47f0 msgr2=0x7f6ea0156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6e9e59c640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea00b47f0 0x7f6ea0156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6e9e59c640 1 -- 172.21.15.16:0/653454176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea014f9e0 con 0x7f6ea006baf0 2024-09-12T01:33:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6e9dd9b640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea00b47f0 0x7f6ea0156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.658+0000 7f6e9ed9d640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6ea015e760 0x7f6ea006f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.659+0000 7f6e9e59c640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ea006baf0 0x7f6ea0150a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000e870 tx=0x7f6e9000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.659+0000 7f6e8effd640 1 -- 172.21.15.16:0/653454176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90017070 con 0x7f6ea006baf0 2024-09-12T01:33:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.659+0000 7f6e8effd640 1 -- 172.21.15.16:0/653454176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e9000c940 con 0x7f6ea006baf0 2024-09-12T01:33:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.659+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/653454176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea0156db0 con 0x7f6ea006baf0 2024-09-12T01:33:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.659+0000 7f6e8effd640 1 -- 172.21.15.16:0/653454176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000cd30 con 0x7f6ea006baf0 2024-09-12T01:33:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.659+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/653454176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea014ff60 con 0x7f6ea006baf0 2024-09-12T01:33:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.660+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/653454176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea006c850 con 0x7f6ea006baf0 2024-09-12T01:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.660+0000 7f6e8effd640 1 -- 172.21.15.16:0/653454176 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e9001a070 con 0x7f6ea006baf0 2024-09-12T01:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.661+0000 7f6e8effd640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6e8006be60 0x7f6e8006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.661+0000 7f6e9dd9b640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6e8006be60 0x7f6e8006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.661+0000 7f6e8effd640 1 -- 172.21.15.16:0/653454176 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6e9008afe0 con 0x7f6ea006baf0 2024-09-12T01:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.661+0000 7f6e9dd9b640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6e8006be60 0x7f6e8006e320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00d19e0 tx=0x7f6e94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.665+0000 7f6e8effd640 1 -- 172.21.15.16:0/653454176 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6e900596e0 con 0x7f6ea006baf0 2024-09-12T01:33:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.841+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/653454176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ea0156650 con 0x7f6ea006baf0 2024-09-12T01:33:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.842+0000 7f6e8effd640 1 -- 172.21.15.16:0/653454176 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18574 (secure 0 0 0) 0x7f6e90058fe0 con 0x7f6ea006baf0 2024-09-12T01:33:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:18.842 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:33:18.213622+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:33:09.364639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:33:10.478237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:33:14.204624+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:33:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.844+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/653454176 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6e8006be60 msgr2=0x7f6e8006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.844+0000 7f6ea4bd8640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6e8006be60 0x7f6e8006e320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00d19e0 tx=0x7f6e94009040 comp rx=0 tx=0).stop 2024-09-12T01:33:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.844+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/653454176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ea006baf0 msgr2=0x7f6ea0150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.844+0000 7f6ea4bd8640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ea006baf0 0x7f6ea0150a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000e870 tx=0x7f6e9000ed30 comp rx=0 tx=0).stop 2024-09-12T01:33:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.844+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/653454176 shutdown_connections 2024-09-12T01:33:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.844+0000 7f6ea4bd8640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6e8006be60 0x7f6e8006e320 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.844+0000 7f6ea4bd8640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6ea015e760 0x7f6ea006f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.844+0000 7f6ea4bd8640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea00b47f0 0x7f6ea0156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.844+0000 7f6ea4bd8640 1 --2- 172.21.15.16:0/653454176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ea006baf0 0x7f6ea0150a40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.844+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/653454176 >> 172.21.15.16:0/653454176 conn(0x7f6ea00b2c50 msgr2=0x7f6ea00ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.844+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/653454176 shutdown_connections 2024-09-12T01:33:18.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:18.844+0000 7f6ea4bd8640 1 -- 172.21.15.16:0/653454176 wait complete. 2024-09-12T01:33:19.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T01:33:11.492995+0000', 'modified': '2024-09-12T01:33:16.541699+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/2179301157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6833', 'nonce': 2179301157}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 2179301157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4287, 'qdb_cluster': [4287]} 2024-09-12T01:33:19.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T01:33:19.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-12T01:33:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.366+0000 7fd49b08d640 1 Processor -- start 2024-09-12T01:33:19.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.366+0000 7fd49b08d640 1 -- start start 2024-09-12T01:33:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd49b08d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4940d4d60 0x7fd4940d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd49b08d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd494162020 0x7fd4940d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd49b08d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd494162500 con 0x7fd49415e660 2024-09-12T01:33:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd49b08d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd4940d2020 con 0x7fd494162020 2024-09-12T01:33:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd49b08d640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fd4941627e0 con 0x7fd4940d4d60 2024-09-12T01:33:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd49962a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd494162020 0x7fd4940d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd498e29640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd49415e660 0x7fd4940d4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58208/0 (socket says 172.21.15.16:58208) 2024-09-12T01:33:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd498e29640 1 -- 172.21.15.16:0/3088427384 learned_addr learned my addr 172.21.15.16:0/3088427384 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd493fff640 1 --2- 172.21.15.16:0/3088427384 >> v2:172.21.15.33:3300/0 conn(0x7fd4940d4d60 0x7fd4940d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd49962a640 1 -- 172.21.15.16:0/3088427384 >> v2:172.21.15.33:3300/0 conn(0x7fd4940d4d60 msgr2=0x7fd4940d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd49962a640 1 --2- 172.21.15.16:0/3088427384 >> v2:172.21.15.33:3300/0 conn(0x7fd4940d4d60 0x7fd4940d24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd49962a640 1 -- 172.21.15.16:0/3088427384 >> v1:172.21.15.16:6789/0 conn(0x7fd49415e660 legacy=0x7fd4940d4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd4937fe640 1 -- 172.21.15.16:0/3088427384 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1115563524 0 0) 0x7fd494162500 con 0x7fd49415e660 2024-09-12T01:33:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd49962a640 1 -- 172.21.15.16:0/3088427384 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd494162ac0 con 0x7fd494162020 2024-09-12T01:33:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.367+0000 7fd493fff640 1 --2- 172.21.15.16:0/3088427384 >> v2:172.21.15.33:3300/0 conn(0x7fd4940d4d60 0x7fd4940d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.368+0000 7fd49962a640 1 --2- 172.21.15.16:0/3088427384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd494162020 0x7fd4940d29f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4940d21a0 tx=0x7fd48c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6dd855417453ea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.368+0000 7fd4937fe640 1 -- 172.21.15.16:0/3088427384 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c031600 con 0x7fd494162020 2024-09-12T01:33:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.368+0000 7fd4937fe640 1 -- 172.21.15.16:0/3088427384 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd48c0317a0 con 0x7fd494162020 2024-09-12T01:33:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.368+0000 7fd4937fe640 1 -- 172.21.15.16:0/3088427384 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c031b70 con 0x7fd494162020 2024-09-12T01:33:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.368+0000 7fd49b08d640 1 -- 172.21.15.16:0/3088427384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd494162020 msgr2=0x7fd4940d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.368+0000 7fd49b08d640 1 --2- 172.21.15.16:0/3088427384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd494162020 0x7fd4940d29f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4940d21a0 tx=0x7fd48c02a5d0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.369+0000 7fd49b08d640 1 -- 172.21.15.16:0/3088427384 shutdown_connections 2024-09-12T01:33:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.369+0000 7fd49b08d640 1 --2- 172.21.15.16:0/3088427384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd494162020 0x7fd4940d29f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.369+0000 7fd49b08d640 1 --2- 172.21.15.16:0/3088427384 >> v2:172.21.15.33:3300/0 conn(0x7fd4940d4d60 0x7fd4940d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.369+0000 7fd49b08d640 1 -- 172.21.15.16:0/3088427384 >> 172.21.15.16:0/3088427384 conn(0x7fd49408de10 msgr2=0x7fd49409d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:19.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.369+0000 7fd49b08d640 1 -- 172.21.15.16:0/3088427384 shutdown_connections 2024-09-12T01:33:19.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.369+0000 7fd49b08d640 1 -- 172.21.15.16:0/3088427384 wait complete. 2024-09-12T01:33:19.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.370+0000 7fd49b08d640 1 Processor -- start 2024-09-12T01:33:19.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.370+0000 7fd49b08d640 1 -- start start 2024-09-12T01:33:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd49b08d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4940d4d60 0x7fd494155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd49b08d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd49415e660 0x7fd49415b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd49b08d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd494162020 0x7fd49415bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd49b08d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4940d4990 con 0x7fd4940d4d60 2024-09-12T01:33:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd49b08d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4940d4810 con 0x7fd494162020 2024-09-12T01:33:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd49b08d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fd4940d4b10 con 0x7fd49415e660 2024-09-12T01:33:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd498e29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4940d4d60 0x7fd494155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd498e29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4940d4d60 0x7fd494155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59078/0 (socket says 172.21.15.16:59078) 2024-09-12T01:33:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd498e29640 1 -- 172.21.15.16:0/111825839 learned_addr learned my addr 172.21.15.16:0/111825839 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd49962a640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd494162020 0x7fd49415bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd493fff640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd49415e660 0x7fd49415b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd498e29640 1 -- 172.21.15.16:0/111825839 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd49415e660 msgr2=0x7fd49415b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd498e29640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd49415e660 0x7fd49415b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd498e29640 1 -- 172.21.15.16:0/111825839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd494162020 msgr2=0x7fd49415bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd498e29640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd494162020 0x7fd49415bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd498e29640 1 -- 172.21.15.16:0/111825839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4941561d0 con 0x7fd4940d4d60 2024-09-12T01:33:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd49962a640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd494162020 0x7fd49415bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd493fff640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd49415e660 0x7fd49415b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd498e29640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4940d4d60 0x7fd494155c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd48400c500 tx=0x7fd48400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd4917fa640 1 -- 172.21.15.16:0/111825839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484018070 con 0x7fd4940d4d60 2024-09-12T01:33:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd4917fa640 1 -- 172.21.15.16:0/111825839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd484007580 con 0x7fd4940d4d60 2024-09-12T01:33:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd49b08d640 1 -- 172.21.15.16:0/111825839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4941e8c10 con 0x7fd4940d4d60 2024-09-12T01:33:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd4917fa640 1 -- 172.21.15.16:0/111825839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484004a20 con 0x7fd4940d4d60 2024-09-12T01:33:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.371+0000 7fd49b08d640 1 -- 172.21.15.16:0/111825839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4941e91d0 con 0x7fd4940d4d60 2024-09-12T01:33:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.373+0000 7fd4917fa640 1 -- 172.21.15.16:0/111825839 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd48401b070 con 0x7fd4940d4d60 2024-09-12T01:33:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.374+0000 7fd4917fa640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd48006be30 0x7fd48006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.374+0000 7fd4917fa640 1 -- 172.21.15.16:0/111825839 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd48408a480 con 0x7fd4940d4d60 2024-09-12T01:33:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.374+0000 7fd493fff640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd48006be30 0x7fd48006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.374+0000 7fd493fff640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd48006be30 0x7fd48006e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd4941f76c0 tx=0x7fd488008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.374+0000 7fd49de2a740 1 Processor -- start 2024-09-12T01:33:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.374+0000 7fd49de2a740 1 -- start start 2024-09-12T01:33:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd49de2a740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf12c0 0x5561f3c36a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd49de2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3beea80 0x5561f3bf16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd49de2a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 0x5561f3beb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd49de2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5561f3b53520 con 0x5561f3beea80 2024-09-12T01:33:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd49de2a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5561f3afb3e0 con 0x5561f3bf2a50 2024-09-12T01:33:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd49de2a740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x5561f3b4f4e0 con 0x5561f3bf12c0 2024-09-12T01:33:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd493fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3beea80 0x5561f3bf16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd493fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3beea80 0x5561f3bf16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59082/0 (socket says 172.21.15.16:59082) 2024-09-12T01:33:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd493fff640 1 -- 172.21.15.16:0/816622983 learned_addr learned my addr 172.21.15.16:0/816622983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd493fff640 1 -- 172.21.15.16:0/816622983 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf12c0 msgr2=0x5561f3c36a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd493fff640 1 --2- 172.21.15.16:0/816622983 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf12c0 0x5561f3c36a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd493fff640 1 -- 172.21.15.16:0/816622983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 msgr2=0x5561f3beb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd49962a640 1 --2- 172.21.15.16:0/816622983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 0x5561f3beb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd493fff640 1 --2- 172.21.15.16:0/816622983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 0x5561f3beb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd493fff640 1 -- 172.21.15.16:0/816622983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f3b162f0 con 0x5561f3beea80 2024-09-12T01:33:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.375+0000 7fd49962a640 1 --2- 172.21.15.16:0/816622983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 0x5561f3beb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.376+0000 7fd493fff640 1 --2- 172.21.15.16:0/816622983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3beea80 0x5561f3bf16a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd48800fac0 tx=0x7fd488012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23180400eefe10f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.376+0000 7fd47a7fc640 1 -- 172.21.15.16:0/816622983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48801b050 con 0x5561f3beea80 2024-09-12T01:33:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.376+0000 7fd47a7fc640 1 -- 172.21.15.16:0/816622983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd488025070 con 0x5561f3beea80 2024-09-12T01:33:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.376+0000 7fd47a7fc640 1 -- 172.21.15.16:0/816622983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488014de0 con 0x5561f3beea80 2024-09-12T01:33:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.376+0000 7fd49de2a740 1 -- 172.21.15.16:0/816622983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3beea80 msgr2=0x5561f3bf16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.376+0000 7fd49de2a740 1 --2- 172.21.15.16:0/816622983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3beea80 0x5561f3bf16a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd48800fac0 tx=0x7fd488012000 comp rx=0 tx=0).stop 2024-09-12T01:33:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.376+0000 7fd49de2a740 1 -- 172.21.15.16:0/816622983 shutdown_connections 2024-09-12T01:33:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.376+0000 7fd49de2a740 1 --2- 172.21.15.16:0/816622983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 0x5561f3beb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.376+0000 7fd49de2a740 1 --2- 172.21.15.16:0/816622983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3beea80 0x5561f3bf16a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.376+0000 7fd49de2a740 1 --2- 172.21.15.16:0/816622983 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf12c0 0x5561f3c36a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.376+0000 7fd49de2a740 1 -- 172.21.15.16:0/816622983 >> 172.21.15.16:0/816622983 conn(0x5561f3beda80 msgr2=0x5561f3c35f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.376+0000 7fd49de2a740 1 -- 172.21.15.16:0/816622983 shutdown_connections 2024-09-12T01:33:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.376+0000 7fd49de2a740 1 -- 172.21.15.16:0/816622983 wait complete. 2024-09-12T01:33:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.377+0000 7fd49de2a740 1 -- start start 2024-09-12T01:33:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.377+0000 7fd49de2a740 1 Processor -- start 2024-09-12T01:33:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd49de2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3bf12c0 0x5561f3c33640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd49de2a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 0x5561f3beb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd49de2a740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3c10d10 0x5561f3c113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd49de2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5561f3becf20 con 0x5561f3bf12c0 2024-09-12T01:33:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd49de2a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5561f3adc450 con 0x5561f3bf2a50 2024-09-12T01:33:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd49de2a740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x5561f3c37290 con 0x5561f3c10d10 2024-09-12T01:33:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd498e29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3bf12c0 0x5561f3c33640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd49de2a740 10 client.0 _ll_register_callbacks cb 0x5561f3865ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:33:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd498e29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3bf12c0 0x5561f3c33640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59092/0 (socket says 172.21.15.16:59092) 2024-09-12T01:33:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd498e29640 1 -- 172.21.15.16:0/1868151005 learned_addr learned my addr 172.21.15.16:0/1868151005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd493fff640 1 --2- 172.21.15.16:0/1868151005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 0x5561f3beb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd498e29640 1 -- 172.21.15.16:0/1868151005 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3c10d10 msgr2=0x5561f3c113b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd498e29640 1 --2- 172.21.15.16:0/1868151005 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3c10d10 0x5561f3c113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd498e29640 1 -- 172.21.15.16:0/1868151005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 msgr2=0x5561f3beb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd498e29640 1 --2- 172.21.15.16:0/1868151005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 0x5561f3beb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd498e29640 1 -- 172.21.15.16:0/1868151005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f38662d0 con 0x5561f3bf12c0 2024-09-12T01:33:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd498e29640 1 --2- 172.21.15.16:0/1868151005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3bf12c0 0x5561f3c33640 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd484020030 tx=0x7fd484058480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3047c3212d98ff5c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.378+0000 7fd4737fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:33:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd4737fe640 1 -- 172.21.15.16:0/1868151005 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48408a270 con 0x5561f3bf12c0 2024-09-12T01:33:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd4737fe640 1 -- 172.21.15.16:0/1868151005 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd484004bc0 con 0x5561f3bf12c0 2024-09-12T01:33:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd49de2a740 1 -- 172.21.15.16:0/1868151005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd488025070 con 0x5561f3bf12c0 2024-09-12T01:33:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd4737fe640 1 -- 172.21.15.16:0/1868151005 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fd484061060 con 0x5561f3bf12c0 2024-09-12T01:33:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd49de2a740 10 client.4401 fetch_fsmap learned FSMap version 6 2024-09-12T01:33:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd49de2a740 1 -- 172.21.15.16:0/1868151005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x5561f39306b0 con 0x5561f3bf12c0 2024-09-12T01:33:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd4737fe640 1 -- 172.21.15.16:0/1868151005 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd488025070 con 0x5561f3bf12c0 2024-09-12T01:33:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd4737fe640 1 -- 172.21.15.16:0/1868151005 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 6) ==== 3510+0+0 (secure 0 0 0) 0x7fd4840701b0 con 0x5561f3bf12c0 2024-09-12T01:33:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd49de2a740 10 client.4401 fetch_fsmap finished waiting for FSMap version 6 2024-09-12T01:33:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd49de2a740 10 client.4401 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T01:33:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd49de2a740 20 client.4401 populate_metadata read hostname 'smithi016' 2024-09-12T01:33:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd49de2a740 1 --2- 172.21.15.16:0/1868151005 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5561f3c13620 0x5561f3c19f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd49de2a740 4 client.4401 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:33:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd49de2a740 1 -- 172.21.15.16:0/1868151005 --> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561f3b60c50 con 0x5561f3c13620 2024-09-12T01:33:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.379+0000 7fd493fff640 1 --2- 172.21.15.16:0/1868151005 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5561f3c13620 0x5561f3c19f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.380+0000 7fd4737fe640 10 client.4401 ms_handle_connect on v2:172.21.15.33:6833/2179301157 2024-09-12T01:33:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.380+0000 7fd493fff640 1 --2- 172.21.15.16:0/1868151005 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5561f3c13620 0x5561f3c19f70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9a345f9160466c4 server_cookie=e956177b00ea7c09 in_seq=0 out_seq=0 2024-09-12T01:33:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd4737fe640 1 -- 172.21.15.16:0/1868151005 <== mds.0 v2:172.21.15.33:6833/2179301157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561f3b60c50 con 0x5561f3c13620 2024-09-12T01:33:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd4737fe640 10 client.4401 handle_command_reply: tid=0 2024-09-12T01:33:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd49de2a740 1 client.4401 shutdown 2024-09-12T01:33:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd49de2a740 1 -- 172.21.15.16:0/1868151005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3bf12c0 msgr2=0x5561f3c33640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd49de2a740 1 --2- 172.21.15.16:0/1868151005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3bf12c0 0x5561f3c33640 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd484020030 tx=0x7fd484058480 comp rx=0 tx=0).stop 2024-09-12T01:33:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd49de2a740 1 -- 172.21.15.16:0/1868151005 shutdown_connections 2024-09-12T01:33:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd49de2a740 1 --2- 172.21.15.16:0/1868151005 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5561f3c13620 0x5561f3c19f70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd49de2a740 1 --2- 172.21.15.16:0/1868151005 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3c10d10 0x5561f3c113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd49de2a740 1 --2- 172.21.15.16:0/1868151005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 0x5561f3beb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd4737fe640 0 client.4401 ms_handle_reset on v2:172.21.15.33:6833/2179301157 2024-09-12T01:33:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd49de2a740 1 --2- 172.21.15.16:0/1868151005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3bf12c0 0x5561f3c33640 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd49de2a740 1 -- 172.21.15.16:0/1868151005 >> 172.21.15.16:0/1868151005 conn(0x5561f3beea80 msgr2=0x5561f3c35690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd49de2a740 1 -- 172.21.15.16:0/1868151005 shutdown_connections 2024-09-12T01:33:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd49de2a740 1 -- 172.21.15.16:0/1868151005 wait complete. 2024-09-12T01:33:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.381+0000 7fd49de2a740 20 client.4401 trim_cache size 0 max 16384 2024-09-12T01:33:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.399+0000 7fd49de2a740 1 Processor -- start 2024-09-12T01:33:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.399+0000 7fd49de2a740 1 -- start start 2024-09-12T01:33:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49de2a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf12c0 0x5561f3caaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49de2a740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf2a50 0x5561f3c101e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49de2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 0x5561f3c10720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49de2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd494200b30 con 0x5561f3c10d10 2024-09-12T01:33:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49de2a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4940ca810 con 0x5561f3bf12c0 2024-09-12T01:33:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49de2a740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fd4942004e0 con 0x5561f3bf2a50 2024-09-12T01:33:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49962a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 0x5561f3c10720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49962a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 0x5561f3c10720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59094/0 (socket says 172.21.15.16:59094) 2024-09-12T01:33:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49962a640 1 -- 172.21.15.16:0/3191046698 learned_addr learned my addr 172.21.15.16:0/3191046698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd493fff640 1 --2- 172.21.15.16:0/3191046698 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf2a50 0x5561f3c101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd498e29640 1 --2- 172.21.15.16:0/3191046698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf12c0 0x5561f3caaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49962a640 1 -- 172.21.15.16:0/3191046698 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf2a50 msgr2=0x5561f3c101e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49962a640 1 --2- 172.21.15.16:0/3191046698 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf2a50 0x5561f3c101e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49962a640 1 -- 172.21.15.16:0/3191046698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf12c0 msgr2=0x5561f3caaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49962a640 1 --2- 172.21.15.16:0/3191046698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf12c0 0x5561f3caaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49962a640 1 -- 172.21.15.16:0/3191046698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f3c59b30 con 0x5561f3c10d10 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.400+0000 7fd49962a640 1 --2- 172.21.15.16:0/3191046698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 0x5561f3c10720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd48c03c4b0 tx=0x7fd48c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65368180a607e0b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.401+0000 7fd479ffb640 1 -- 172.21.15.16:0/3191046698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c049070 con 0x5561f3c10d10 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.401+0000 7fd479ffb640 1 -- 172.21.15.16:0/3191046698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd48c03c810 con 0x5561f3c10d10 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.401+0000 7fd479ffb640 1 -- 172.21.15.16:0/3191046698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c03cbc0 con 0x5561f3c10d10 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.401+0000 7fd49de2a740 1 -- 172.21.15.16:0/3191046698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 msgr2=0x5561f3c10720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.401+0000 7fd49de2a740 1 --2- 172.21.15.16:0/3191046698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 0x5561f3c10720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd48c03c4b0 tx=0x7fd48c02c000 comp rx=0 tx=0).stop 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.401+0000 7fd49de2a740 1 -- 172.21.15.16:0/3191046698 shutdown_connections 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.401+0000 7fd49de2a740 1 --2- 172.21.15.16:0/3191046698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 0x5561f3c10720 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.401+0000 7fd49de2a740 1 --2- 172.21.15.16:0/3191046698 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf2a50 0x5561f3c101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.401+0000 7fd49de2a740 1 --2- 172.21.15.16:0/3191046698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf12c0 0x5561f3caaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.401+0000 7fd49de2a740 1 -- 172.21.15.16:0/3191046698 >> 172.21.15.16:0/3191046698 conn(0x5561f3beea80 msgr2=0x5561f3c59d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.401+0000 7fd49de2a740 1 -- 172.21.15.16:0/3191046698 shutdown_connections 2024-09-12T01:33:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.401+0000 7fd49de2a740 1 -- 172.21.15.16:0/3191046698 wait complete. 2024-09-12T01:33:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.402+0000 7fd49de2a740 1 -- start start 2024-09-12T01:33:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.402+0000 7fd49de2a740 1 Processor -- start 2024-09-12T01:33:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49de2a740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf12c0 0x5561f3ca90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49de2a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 0x5561f3bea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49de2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 0x5561f3bead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49de2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5561f3c1afc0 con 0x5561f3c10d10 2024-09-12T01:33:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49de2a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5561f3bebf60 con 0x5561f3bf2a50 2024-09-12T01:33:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49de2a740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x5561f3c13a60 con 0x5561f3bf12c0 2024-09-12T01:33:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49de2a740 10 client.0 _ll_register_callbacks cb 0x7fd484001060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:33:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49962a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 0x5561f3bead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49962a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 0x5561f3bead10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59100/0 (socket says 172.21.15.16:59100) 2024-09-12T01:33:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49962a640 1 -- 172.21.15.16:0/3872369107 learned_addr learned my addr 172.21.15.16:0/3872369107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd498e29640 1 --2- 172.21.15.16:0/3872369107 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf12c0 0x5561f3ca90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd493fff640 1 --2- 172.21.15.16:0/3872369107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 0x5561f3bea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49962a640 1 -- 172.21.15.16:0/3872369107 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf12c0 msgr2=0x5561f3ca90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49962a640 1 --2- 172.21.15.16:0/3872369107 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf12c0 0x5561f3ca90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49962a640 1 -- 172.21.15.16:0/3872369107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 msgr2=0x5561f3bea7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49962a640 1 --2- 172.21.15.16:0/3872369107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 0x5561f3bea7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49962a640 1 -- 172.21.15.16:0/3872369107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f3c45b70 con 0x5561f3c10d10 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49962a640 1 --2- 172.21.15.16:0/3872369107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 0x5561f3bead10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009e50 tx=0x7fd48c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01a14323fe37685 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd4737fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd4737fe640 1 -- 172.21.15.16:0/3872369107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c049070 con 0x5561f3c10d10 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd4737fe640 1 -- 172.21.15.16:0/3872369107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd48c003da0 con 0x5561f3c10d10 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49de2a740 1 -- 172.21.15.16:0/3872369107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd48c03c810 con 0x5561f3c10d10 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd4737fe640 1 -- 172.21.15.16:0/3872369107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fd48c004040 con 0x5561f3c10d10 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49de2a740 10 client.4407 fetch_fsmap learned FSMap version 6 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.403+0000 7fd49de2a740 1 -- 172.21.15.16:0/3872369107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x5561f3ca9600 con 0x5561f3c10d10 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.404+0000 7fd4737fe640 1 -- 172.21.15.16:0/3872369107 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd48c03c810 con 0x5561f3c10d10 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.404+0000 7fd4737fe640 1 -- 172.21.15.16:0/3872369107 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 6) ==== 3510+0+0 (secure 0 0 0) 0x7fd48c043030 con 0x5561f3c10d10 2024-09-12T01:33:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.404+0000 7fd49de2a740 10 client.4407 fetch_fsmap finished waiting for FSMap version 6 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.404+0000 7fd49de2a740 10 client.4407 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.404+0000 7fd49de2a740 20 client.4407 populate_metadata read hostname 'smithi016' 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.404+0000 7fd49de2a740 1 --2- 172.21.15.16:0/3872369107 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5561f3c16ef0 0x5561f3cb3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.404+0000 7fd49de2a740 4 client.4407 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.404+0000 7fd49de2a740 1 -- 172.21.15.16:0/3872369107 --> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] -- command(tid 0: {"prefix": "status"}) -- 0x5561f3b5afa0 con 0x5561f3c16ef0 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.404+0000 7fd498e29640 1 --2- 172.21.15.16:0/3872369107 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5561f3c16ef0 0x5561f3cb3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.404+0000 7fd498e29640 1 --2- 172.21.15.16:0/3872369107 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5561f3c16ef0 0x5561f3cb3e40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c74068f88e58de34 server_cookie=e529c6411e8a4efd in_seq=0 out_seq=0 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.404+0000 7fd4737fe640 10 client.4407 ms_handle_connect on v2:172.21.15.33:6833/2179301157 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd4737fe640 1 -- 172.21.15.16:0/3872369107 <== mds.0 v2:172.21.15.33:6833/2179301157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5561f3b5afa0 con 0x5561f3c16ef0 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd4737fe640 10 client.4407 handle_command_reply: tid=0 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49de2a740 1 client.4407 shutdown 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49de2a740 1 -- 172.21.15.16:0/3872369107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 msgr2=0x5561f3bead10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49de2a740 1 --2- 172.21.15.16:0/3872369107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 0x5561f3bead10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009e50 tx=0x7fd48c004ab0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49de2a740 1 -- 172.21.15.16:0/3872369107 shutdown_connections 2024-09-12T01:33:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49de2a740 1 --2- 172.21.15.16:0/3872369107 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5561f3c16ef0 0x5561f3cb3e40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49de2a740 1 --2- 172.21.15.16:0/3872369107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561f3c10d10 0x5561f3bead10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49de2a740 1 --2- 172.21.15.16:0/3872369107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561f3bf2a50 0x5561f3bea7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd4737fe640 0 client.4407 ms_handle_reset on v2:172.21.15.33:6833/2179301157 2024-09-12T01:33:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49de2a740 1 --2- 172.21.15.16:0/3872369107 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5561f3bf12c0 0x5561f3ca90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49de2a740 1 -- 172.21.15.16:0/3872369107 >> 172.21.15.16:0/3872369107 conn(0x5561f3beea80 msgr2=0x5561f3c101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49de2a740 1 -- 172.21.15.16:0/3872369107 shutdown_connections 2024-09-12T01:33:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49de2a740 1 -- 172.21.15.16:0/3872369107 wait complete. 2024-09-12T01:33:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49de2a740 20 client.4407 trim_cache size 0 max 16384 2024-09-12T01:33:19.412 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:33:19.413 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "cb5903e4-2f7f-485b-9d92-173a6afaa2f8", 2024-09-12T01:33:19.413 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-12T01:33:19.413 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4287, 2024-09-12T01:33:19.413 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T01:33:19.413 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T01:33:19.413 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T01:33:19.413 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 7.2557813009999998, 2024-09-12T01:33:19.413 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 6, 2024-09-12T01:33:19.413 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 26, 2024-09-12T01:33:19.413 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 25, 2024-09-12T01:33:19.413 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 13.279259768999999 2024-09-12T01:33:19.413 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:33:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49b08d640 1 -- 172.21.15.16:0/111825839 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd48006be30 msgr2=0x7fd48006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49b08d640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd48006be30 0x7fd48006e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd4941f76c0 tx=0x7fd488008040 comp rx=0 tx=0).stop 2024-09-12T01:33:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49b08d640 1 -- 172.21.15.16:0/111825839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4940d4d60 msgr2=0x7fd494155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49b08d640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4940d4d60 0x7fd494155c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd48400c500 tx=0x7fd48400c9c0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49b08d640 1 -- 172.21.15.16:0/111825839 shutdown_connections 2024-09-12T01:33:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49b08d640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd48006be30 0x7fd48006e2f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49b08d640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd494162020 0x7fd49415bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49b08d640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd49415e660 0x7fd49415b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49b08d640 1 --2- 172.21.15.16:0/111825839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4940d4d60 0x7fd494155c90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.405+0000 7fd49b08d640 1 -- 172.21.15.16:0/111825839 >> 172.21.15.16:0/111825839 conn(0x7fd49408de10 msgr2=0x7fd4940744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.406+0000 7fd49b08d640 1 -- 172.21.15.16:0/111825839 shutdown_connections 2024-09-12T01:33:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.406+0000 7fd49b08d640 1 -- 172.21.15.16:0/111825839 wait complete. 2024-09-12T01:33:19.416 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T01:33:19.416 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:33:19.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T01:33:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.519+0000 7ff8038ae640 1 Processor -- start 2024-09-12T01:33:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.519+0000 7ff8038ae640 1 -- start start 2024-09-12T01:33:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff8038ae640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7fc15e800 0x7ff7fc0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff8038ae640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7fc0d3470 0x7ff7fc0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff8038ae640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7fc0cfd30 con 0x7ff7fc0cf910 2024-09-12T01:33:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff8038ae640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff7fc15ebe0 con 0x7ff7fc0d3470 2024-09-12T01:33:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff8038ae640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff7fc0d42d0 con 0x7ff7fc15e800 2024-09-12T01:33:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff801e4b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7fc0d3470 0x7ff7fc0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff800e49640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff7fc0cf910 0x7ff7fc06d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58218/0 (socket says 172.21.15.16:58218) 2024-09-12T01:33:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff800e49640 1 -- 172.21.15.16:0/3069285082 learned_addr learned my addr 172.21.15.16:0/3069285082 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff80164a640 1 --2- 172.21.15.16:0/3069285082 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7fc15e800 0x7ff7fc0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff801e4b640 1 -- 172.21.15.16:0/3069285082 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7fc15e800 msgr2=0x7ff7fc0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff801e4b640 1 --2- 172.21.15.16:0/3069285082 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7fc15e800 0x7ff7fc0cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff801e4b640 1 -- 172.21.15.16:0/3069285082 >> v1:172.21.15.16:6789/0 conn(0x7ff7fc0cf910 legacy=0x7ff7fc06d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff801e4b640 1 -- 172.21.15.16:0/3069285082 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7fc0d3ff0 con 0x7ff7fc0d3470 2024-09-12T01:33:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.520+0000 7ff7f3fff640 1 -- 172.21.15.16:0/3069285082 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3061857831 0 0) 0x7ff7fc0cfd30 con 0x7ff7fc0cf910 2024-09-12T01:33:19.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.521+0000 7ff801e4b640 1 --2- 172.21.15.16:0/3069285082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7fc0d3470 0x7ff7fc0d3920 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009d90 tx=0x7ff7f402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2bced205820be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:19.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.521+0000 7ff7f3fff640 1 -- 172.21.15.16:0/3069285082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4007740 con 0x7ff7fc0d3470 2024-09-12T01:33:19.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.521+0000 7ff7f3fff640 1 -- 172.21.15.16:0/3069285082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f40078e0 con 0x7ff7fc0d3470 2024-09-12T01:33:19.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.521+0000 7ff7f3fff640 1 -- 172.21.15.16:0/3069285082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4007cb0 con 0x7ff7fc0d3470 2024-09-12T01:33:19.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.522+0000 7ff8038ae640 1 -- 172.21.15.16:0/3069285082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7fc0d3470 msgr2=0x7ff7fc0d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.522+0000 7ff8038ae640 1 --2- 172.21.15.16:0/3069285082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7fc0d3470 0x7ff7fc0d3920 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009d90 tx=0x7ff7f402a660 comp rx=0 tx=0).stop 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.522+0000 7ff8038ae640 1 -- 172.21.15.16:0/3069285082 shutdown_connections 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.522+0000 7ff8038ae640 1 --2- 172.21.15.16:0/3069285082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7fc0d3470 0x7ff7fc0d3920 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.522+0000 7ff8038ae640 1 --2- 172.21.15.16:0/3069285082 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7fc15e800 0x7ff7fc0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.522+0000 7ff8038ae640 1 -- 172.21.15.16:0/3069285082 >> 172.21.15.16:0/3069285082 conn(0x7ff7fc09d750 msgr2=0x7ff7fc0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.522+0000 7ff8038ae640 1 -- 172.21.15.16:0/3069285082 shutdown_connections 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.522+0000 7ff8038ae640 1 -- 172.21.15.16:0/3069285082 wait complete. 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.523+0000 7ff8038ae640 1 Processor -- start 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.523+0000 7ff8038ae640 1 -- start start 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff8038ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7fc0cf910 0x7ff7fc0cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff8038ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7fc0d3470 0x7ff7fc157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff8038ae640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7fc15e800 0x7ff7fc0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff8038ae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc06d170 con 0x7ff7fc0cf910 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff8038ae640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc06ce90 con 0x7ff7fc0d3470 2024-09-12T01:33:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff8038ae640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff7fc0d4e30 con 0x7ff7fc15e800 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff80164a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7fc0cf910 0x7ff7fc0cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff80164a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7fc0cf910 0x7ff7fc0cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59116/0 (socket says 172.21.15.16:59116) 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff80164a640 1 -- 172.21.15.16:0/2728587772 learned_addr learned my addr 172.21.15.16:0/2728587772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff801e4b640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7fc15e800 0x7ff7fc0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff800e49640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7fc0d3470 0x7ff7fc157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff80164a640 1 -- 172.21.15.16:0/2728587772 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7fc15e800 msgr2=0x7ff7fc0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff80164a640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7fc15e800 0x7ff7fc0ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff80164a640 1 -- 172.21.15.16:0/2728587772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7fc0d3470 msgr2=0x7ff7fc157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff80164a640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7fc0d3470 0x7ff7fc157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff80164a640 1 -- 172.21.15.16:0/2728587772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7fc0b7350 con 0x7ff7fc0cf910 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.524+0000 7ff80164a640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7fc0cf910 0x7ff7fc0cada0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800d620 tx=0x7ff7f800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.525+0000 7ff7f1ffb640 1 -- 172.21.15.16:0/2728587772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800b620 con 0x7ff7fc0cf910 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.525+0000 7ff7f1ffb640 1 -- 172.21.15.16:0/2728587772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f8019070 con 0x7ff7fc0cf910 2024-09-12T01:33:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.525+0000 7ff8038ae640 1 -- 172.21.15.16:0/2728587772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7fc0c83b0 con 0x7ff7fc0cf910 2024-09-12T01:33:19.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.525+0000 7ff7f1ffb640 1 -- 172.21.15.16:0/2728587772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800ba00 con 0x7ff7fc0cf910 2024-09-12T01:33:19.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.525+0000 7ff8038ae640 1 -- 172.21.15.16:0/2728587772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7fc0c8970 con 0x7ff7fc0cf910 2024-09-12T01:33:19.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.526+0000 7ff8038ae640 1 -- 172.21.15.16:0/2728587772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7fc068880 con 0x7ff7fc0cf910 2024-09-12T01:33:19.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.526+0000 7ff7f1ffb640 1 -- 172.21.15.16:0/2728587772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff7f800bba0 con 0x7ff7fc0cf910 2024-09-12T01:33:19.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.527+0000 7ff7f1ffb640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff7e006be30 0x7ff7e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:19.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.527+0000 7ff7f1ffb640 1 -- 172.21.15.16:0/2728587772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff7f808b450 con 0x7ff7fc0cf910 2024-09-12T01:33:19.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.527+0000 7ff800e49640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff7e006be30 0x7ff7e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:19.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.527+0000 7ff800e49640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff7e006be30 0x7ff7e006e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec007f70 tx=0x7ff7ec007f00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:19.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.531+0000 7ff7f1ffb640 1 -- 172.21.15.16:0/2728587772 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff7fc0b7350 con 0x7ff7fc0cf910 2024-09-12T01:33:19.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.741+0000 7ff8038ae640 1 -- 172.21.15.16:0/2728587772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff7fc0b5590 con 0x7ff7fc0cf910 2024-09-12T01:33:19.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.742+0000 7ff7f1ffb640 1 -- 172.21.15.16:0/2728587772 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7ff7f8059470 con 0x7ff7fc0cf910 2024-09-12T01:33:19.741 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T01:33:19.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.743+0000 7ff8038ae640 1 -- 172.21.15.16:0/2728587772 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff7e006be30 msgr2=0x7ff7e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.743+0000 7ff8038ae640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff7e006be30 0x7ff7e006e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec007f70 tx=0x7ff7ec007f00 comp rx=0 tx=0).stop 2024-09-12T01:33:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.743+0000 7ff8038ae640 1 -- 172.21.15.16:0/2728587772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7fc0cf910 msgr2=0x7ff7fc0cada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.743+0000 7ff8038ae640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7fc0cf910 0x7ff7fc0cada0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800d620 tx=0x7ff7f800dae0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.744+0000 7ff8038ae640 1 -- 172.21.15.16:0/2728587772 shutdown_connections 2024-09-12T01:33:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.744+0000 7ff8038ae640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff7e006be30 0x7ff7e006e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.744+0000 7ff8038ae640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7fc15e800 0x7ff7fc0ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.744+0000 7ff8038ae640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7fc0d3470 0x7ff7fc157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.744+0000 7ff8038ae640 1 --2- 172.21.15.16:0/2728587772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7fc0cf910 0x7ff7fc0cada0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.744+0000 7ff8038ae640 1 -- 172.21.15.16:0/2728587772 >> 172.21.15.16:0/2728587772 conn(0x7ff7fc09d750 msgr2=0x7ff7fc0b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.744+0000 7ff8038ae640 1 -- 172.21.15.16:0/2728587772 shutdown_connections 2024-09-12T01:33:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19.744+0000 7ff8038ae640 1 -- 172.21.15.16:0/2728587772 wait complete. 2024-09-12T01:33:19.754 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T01:33:19.754 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T01:33:19.754 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-12T01:33:19.754 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-12T01:33:19.754 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:33:19.754 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T01:33:19.754 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T01:33:19.755 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T01:33:19.755 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-12T01:33:19.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:19.755 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-12T01:33:19.771 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T01:33:19.771 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T01:33:19.771 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T01:33:19.771 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:19.771 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-12T01:33:19.771 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:19.771 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-12T01:33:19.772 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-12T01:33:19.772 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T01:33:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-12T01:33:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T01:33:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42762sec preferred_lft 42762sec 2024-09-12T01:33:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-12T01:33:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:19.772 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:19.773 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:19.773 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-12T01:33:19.773 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T01:33:19.774 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:19.774 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:19.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-12T01:33:19.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-12T01:33:19.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-12T01:33:19.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T01:33:19.774 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:19.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:20.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:20.009 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T01:33:20.035 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-12T01:33:20.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:20.037 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-12T01:33:20.105 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-12T01:33:20.106 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T01:33:20.106 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T01:33:20.106 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T01:33:20.106 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T01:33:20.106 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:20.106 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:20.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T01:33:20.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T01:33:20.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T01:33:20.107 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:20.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:20.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:20.418 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-12T01:33:20.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:20.435 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-12T01:33:20.490 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:20.491 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:20.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:33:20.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-12T01:33:20.491 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:20.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:20.613 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-12T01:33:20.613 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:20.614 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:20.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-12T01:33:20.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T01:33:20.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-12T01:33:20.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-12T01:33:20.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-12T01:33:20.615 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:20.757 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:20.758 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:20.758 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-12T01:33:20.758 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T01:33:20.758 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:20.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:20.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:20.837 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-12T01:33:20.837 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:33:20.837 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:33:20.853 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-12T01:33:20.854 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:33:20.854 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:33:20.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-12T01:33:20.976 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:33:21.032 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-12T01:33:21.032 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-12T01:33:21.032 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-12T01:33:21.032 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-12T01:33:21.032 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-12T01:33:21.032 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-12T01:33:21.032 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-12T01:33:21.032 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-12T01:33:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-12T01:33:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-12T01:33:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-12T01:33:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-12T01:33:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:33:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-12T01:33:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-12T01:33:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-12T01:33:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-12T01:33:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-12T01:33:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-12T01:33:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:33:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:33:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:33:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:33:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-12T01:33:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-12T01:33:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T01:33:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T01:33:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T01:33:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T01:33:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-12T01:33:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:33:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:33:21.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:21.035 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-12T01:33:21.088 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T01:33:21.088 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-12T01:33:21.130 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-12T01:33:21.156 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:33:21.214 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-12T01:33:21.214 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-12T01:33:21.214 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-12T01:33:21.214 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-12T01:33:21.214 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-12T01:33:21.214 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-12T01:33:21.214 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-12T01:33:21.214 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-12T01:33:21.214 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-12T01:33:21.214 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-12T01:33:21.214 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-12T01:33:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-12T01:33:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:33:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-12T01:33:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-12T01:33:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-12T01:33:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-12T01:33:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-12T01:33:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-12T01:33:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:33:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:33:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:33:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:33:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-12T01:33:21.216 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-12T01:33:21.216 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T01:33:21.216 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T01:33:21.216 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T01:33:21.216 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T01:33:21.216 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-12T01:33:21.216 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:33:21.216 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:33:21.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:21.216 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-12T01:33:21.220 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:2024-09-12T01:33:21.220+0000 7faff4b22480 -1 init, newargv = 0x5609890a7100 newargc=15 2024-09-12T01:33:21.220 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[41984]: starting ceph client 2024-09-12T01:33:21.227 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[41984]: starting fuse 2024-09-12T01:33:21.237 INFO:teuthology.orchestra.run.smithi016.stdout:43 2024-09-12T01:33:21.237 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-09-12T01:33:21.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-12T01:33:21.237 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-12T01:33:21.238 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:33:21.239 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-12T01:33:21.239 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T01:33:21.239 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T01:33:21.239 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T01:33:21.239 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-12T01:33:21.239 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-12T01:33:21.239 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-12T01:33:21.239 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T01:33:21.239 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:33:21.239 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.admin")) 2024-09-12T01:33:21.239 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-12T01:33:21.359 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.admin.41984.asok 2024-09-12T01:33:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:21.369 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41984.asok 2024-09-12T01:33:21.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:21.369 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41984.asok status 2024-09-12T01:33:21.506 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:33:21.506 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-12T01:33:21.506 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T01:33:21.506 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:33:21.506 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "admin", 2024-09-12T01:33:21.506 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-12T01:33:21.506 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-12T01:33:21.506 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "41984", 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4425, 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4425 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-12T01:33:21.507 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4257260345 2024-09-12T01:33:21.508 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:33:21.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:33:21.508 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-12T01:33:21.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-12T01:33:21.508 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-12T01:33:21.508 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4257260345 2024-09-12T01:33:21.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:33:21.508 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4425 192.168.0.1:0/4257260345", 2024-09-12T01:33:21.508 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/4257260345", 2024-09-12T01:33:21.508 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-12T01:33:21.508 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 6, 2024-09-12T01:33:21.508 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 26, 2024-09-12T01:33:21.509 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-12T01:33:21.509 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-12T01:33:21.509 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-12T01:33:21.509 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:33:21.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:21.514 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-12T01:33:21.514 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:33:21.530 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-12T01:33:21.531 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:33:21.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:21.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:33:21.597 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-12T01:33:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T01:33:21.710 INFO:teuthology.orchestra.run.smithi016.stdout:# file: . 2024-09-12T01:33:21.710 INFO:teuthology.orchestra.run.smithi016.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-12T01:33:21.710 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:21.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:21.716 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:33:21.717 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:21.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T01:33:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.876+0000 7fa53372c640 1 Processor -- start 2024-09-12T01:33:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.876+0000 7fa53372c640 1 -- start start 2024-09-12T01:33:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.877+0000 7fa53372c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa52c0d0cc0 0x7fa52c0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.877+0000 7fa53372c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa52c161dc0 con 0x7fa52c0b76c0 2024-09-12T01:33:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.877+0000 7fa53372c640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa52c0d3310 con 0x7fa52c1619e0 2024-09-12T01:33:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.877+0000 7fa53372c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa52c0d3f20 con 0x7fa52c0d0cc0 2024-09-12T01:33:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.877+0000 7fa530cc7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa52c0d0cc0 0x7fa52c0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.877+0000 7fa531cc9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa52c0b76c0 0x7fa52c0b7b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58226/0 (socket says 172.21.15.16:58226) 2024-09-12T01:33:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.877+0000 7fa531cc9640 1 -- 172.21.15.16:0/325272036 learned_addr learned my addr 172.21.15.16:0/325272036 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.877+0000 7fa530cc7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa52c0d0cc0 0x7fa52c0d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59128/0 (socket says 172.21.15.16:59128) 2024-09-12T01:33:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.877+0000 7fa530cc7640 1 -- 172.21.15.16:0/325272036 >> v1:172.21.15.33:6789/0 conn(0x7fa52c1619e0 legacy=0x7fa52c0d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.877+0000 7fa530cc7640 1 -- 172.21.15.16:0/325272036 >> v1:172.21.15.16:6789/0 conn(0x7fa52c0b76c0 legacy=0x7fa52c0b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.877+0000 7fa530cc7640 1 -- 172.21.15.16:0/325272036 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c0d35f0 con 0x7fa52c0d0cc0 2024-09-12T01:33:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.877+0000 7fa523fff640 1 -- 172.21.15.16:0/325272036 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4201410319 0 0) 0x7fa52c161dc0 con 0x7fa52c0b76c0 2024-09-12T01:33:21.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.878+0000 7fa530cc7640 1 --2- 172.21.15.16:0/325272036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c0d0cc0 0x7fa52c0d1100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097a0 tx=0x7fa51c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f49548d718f041 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.878+0000 7fa523fff640 1 -- 172.21.15.16:0/325272036 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c037070 con 0x7fa52c0d0cc0 2024-09-12T01:33:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.878+0000 7fa523fff640 1 -- 172.21.15.16:0/325272036 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa51c032440 con 0x7fa52c0d0cc0 2024-09-12T01:33:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.878+0000 7fa523fff640 1 -- 172.21.15.16:0/325272036 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c032810 con 0x7fa52c0d0cc0 2024-09-12T01:33:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.878+0000 7fa53372c640 1 -- 172.21.15.16:0/325272036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c0d0cc0 msgr2=0x7fa52c0d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.878+0000 7fa53372c640 1 --2- 172.21.15.16:0/325272036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c0d0cc0 0x7fa52c0d1100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097a0 tx=0x7fa51c02a1e0 comp rx=0 tx=0).stop 2024-09-12T01:33:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.879+0000 7fa53372c640 1 -- 172.21.15.16:0/325272036 shutdown_connections 2024-09-12T01:33:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.879+0000 7fa53372c640 1 --2- 172.21.15.16:0/325272036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c0d0cc0 0x7fa52c0d1100 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.879+0000 7fa53372c640 1 -- 172.21.15.16:0/325272036 >> 172.21.15.16:0/325272036 conn(0x7fa52c06d1c0 msgr2=0x7fa52c073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.879+0000 7fa53372c640 1 -- 172.21.15.16:0/325272036 shutdown_connections 2024-09-12T01:33:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.879+0000 7fa53372c640 1 -- 172.21.15.16:0/325272036 wait complete. 2024-09-12T01:33:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.880+0000 7fa53372c640 1 Processor -- start 2024-09-12T01:33:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.880+0000 7fa53372c640 1 -- start start 2024-09-12T01:33:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa53372c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa52c0b76c0 0x7fa52c158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa53372c640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa52c0d0cc0 0x7fa52c159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa53372c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c1619e0 0x7fa52c15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa53372c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c0d1640 con 0x7fa52c1619e0 2024-09-12T01:33:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa53372c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c0d3bb0 con 0x7fa52c0b76c0 2024-09-12T01:33:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa53372c640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa52c0d2250 con 0x7fa52c0d0cc0 2024-09-12T01:33:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa531cc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c1619e0 0x7fa52c15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa531cc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c1619e0 0x7fa52c15bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59144/0 (socket says 172.21.15.16:59144) 2024-09-12T01:33:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa531cc9640 1 -- 172.21.15.16:0/62668985 learned_addr learned my addr 172.21.15.16:0/62668985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa5314c8640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa52c0b76c0 0x7fa52c158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa530cc7640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa52c0d0cc0 0x7fa52c159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa531cc9640 1 -- 172.21.15.16:0/62668985 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa52c0d0cc0 msgr2=0x7fa52c159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa531cc9640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa52c0d0cc0 0x7fa52c159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa531cc9640 1 -- 172.21.15.16:0/62668985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa52c0b76c0 msgr2=0x7fa52c158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa531cc9640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa52c0b76c0 0x7fa52c158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa531cc9640 1 -- 172.21.15.16:0/62668985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c1585a0 con 0x7fa52c1619e0 2024-09-12T01:33:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa531cc9640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c1619e0 0x7fa52c15bdc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa52400c7d0 tx=0x7fa52400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa521ffb640 1 -- 172.21.15.16:0/62668985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524018070 con 0x7fa52c1619e0 2024-09-12T01:33:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa521ffb640 1 -- 172.21.15.16:0/62668985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52400a640 con 0x7fa52c1619e0 2024-09-12T01:33:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa53372c640 1 -- 172.21.15.16:0/62668985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c15b000 con 0x7fa52c1619e0 2024-09-12T01:33:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.881+0000 7fa521ffb640 1 -- 172.21.15.16:0/62668985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52400aa70 con 0x7fa52c1619e0 2024-09-12T01:33:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.882+0000 7fa53372c640 1 -- 172.21.15.16:0/62668985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c15b5c0 con 0x7fa52c1619e0 2024-09-12T01:33:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.883+0000 7fa53372c640 1 -- 172.21.15.16:0/62668985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52c0d1100 con 0x7fa52c1619e0 2024-09-12T01:33:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.883+0000 7fa521ffb640 1 -- 172.21.15.16:0/62668985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa52401b070 con 0x7fa52c1619e0 2024-09-12T01:33:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.883+0000 7fa521ffb640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa51006be30 0x7fa51006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.884+0000 7fa521ffb640 1 -- 172.21.15.16:0/62668985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa52408a720 con 0x7fa52c1619e0 2024-09-12T01:33:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.884+0000 7fa5314c8640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa51006be30 0x7fa51006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.884+0000 7fa5314c8640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa51006be30 0x7fa51006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa528006fd0 tx=0x7fa528008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:21.888+0000 7fa521ffb640 1 -- 172.21.15.16:0/62668985 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa52c1585a0 con 0x7fa52c1619e0 2024-09-12T01:33:22.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.075+0000 7fa53372c640 1 -- 172.21.15.16:0/62668985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa52c0b7ea0 con 0x7fa52c1619e0 2024-09-12T01:33:22.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.075+0000 7fa521ffb640 1 -- 172.21.15.16:0/62668985 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fa524058e20 con 0x7fa52c1619e0 2024-09-12T01:33:22.075 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-12T01:33:22.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.077+0000 7fa53372c640 1 -- 172.21.15.16:0/62668985 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa51006be30 msgr2=0x7fa51006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:22.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.077+0000 7fa53372c640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa51006be30 0x7fa51006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa528006fd0 tx=0x7fa528008040 comp rx=0 tx=0).stop 2024-09-12T01:33:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.078+0000 7fa53372c640 1 -- 172.21.15.16:0/62668985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c1619e0 msgr2=0x7fa52c15bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.078+0000 7fa53372c640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c1619e0 0x7fa52c15bdc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa52400c7d0 tx=0x7fa52400cc90 comp rx=0 tx=0).stop 2024-09-12T01:33:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.078+0000 7fa53372c640 1 -- 172.21.15.16:0/62668985 shutdown_connections 2024-09-12T01:33:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.078+0000 7fa53372c640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa51006be30 0x7fa51006e2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.078+0000 7fa53372c640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c1619e0 0x7fa52c15bdc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.078+0000 7fa53372c640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa52c0d0cc0 0x7fa52c159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.078+0000 7fa53372c640 1 --2- 172.21.15.16:0/62668985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa52c0b76c0 0x7fa52c158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.078+0000 7fa53372c640 1 -- 172.21.15.16:0/62668985 >> 172.21.15.16:0/62668985 conn(0x7fa52c06d1c0 msgr2=0x7fa52c15ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.078+0000 7fa53372c640 1 -- 172.21.15.16:0/62668985 shutdown_connections 2024-09-12T01:33:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.078+0000 7fa53372c640 1 -- 172.21.15.16:0/62668985 wait complete. 2024-09-12T01:33:22.088 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:33:22.088 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:22.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T01:33:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.248+0000 7f0b61bcf640 1 Processor -- start 2024-09-12T01:33:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.248+0000 7f0b61bcf640 1 -- start start 2024-09-12T01:33:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.249+0000 7f0b61bcf640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0b5c15e660 0x7f0b5c0d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.249+0000 7f0b61bcf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0b5c0d4d60 0x7f0b5c0d4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.249+0000 7f0b61bcf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b5c162500 con 0x7f0b5c162020 2024-09-12T01:33:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.249+0000 7f0b61bcf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0b5c0d2020 con 0x7f0b5c0d4d60 2024-09-12T01:33:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.249+0000 7f0b61bcf640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f0b5c1627e0 con 0x7f0b5c15e660 2024-09-12T01:33:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.249+0000 7f0b5affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0b5c0d4d60 0x7f0b5c0d4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.249+0000 7f0b5bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0b5c162020 0x7f0b5c0b45c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58240/0 (socket says 172.21.15.16:58240) 2024-09-12T01:33:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.249+0000 7f0b5bfff640 1 -- 172.21.15.16:0/1789759176 learned_addr learned my addr 172.21.15.16:0/1789759176 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.249+0000 7f0b5b7fe640 1 --2- 172.21.15.16:0/1789759176 >> v2:172.21.15.33:3300/0 conn(0x7f0b5c15e660 0x7f0b5c0d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.250+0000 7f0b5affd640 1 -- 172.21.15.16:0/1789759176 >> v2:172.21.15.33:3300/0 conn(0x7f0b5c15e660 msgr2=0x7f0b5c0d4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.250+0000 7f0b5affd640 1 --2- 172.21.15.16:0/1789759176 >> v2:172.21.15.33:3300/0 conn(0x7f0b5c15e660 0x7f0b5c0d4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.250+0000 7f0b5affd640 1 -- 172.21.15.16:0/1789759176 >> v1:172.21.15.16:6789/0 conn(0x7f0b5c162020 legacy=0x7f0b5c0b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.250+0000 7f0b5a7fc640 1 -- 172.21.15.16:0/1789759176 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2164934681 0 0) 0x7f0b5c162500 con 0x7f0b5c162020 2024-09-12T01:33:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.250+0000 7f0b5affd640 1 -- 172.21.15.16:0/1789759176 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5c162ac0 con 0x7f0b5c0d4d60 2024-09-12T01:33:22.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.250+0000 7f0b5affd640 1 --2- 172.21.15.16:0/1789759176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b5c0d4d60 0x7f0b5c0d4640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0d21a0 tx=0x7f0b5002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f344b3927d1500 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.251+0000 7f0b5a7fc640 1 -- 172.21.15.16:0/1789759176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50037070 con 0x7f0b5c0d4d60 2024-09-12T01:33:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.251+0000 7f0b5a7fc640 1 -- 172.21.15.16:0/1789759176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b50032440 con 0x7f0b5c0d4d60 2024-09-12T01:33:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.251+0000 7f0b5a7fc640 1 -- 172.21.15.16:0/1789759176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5002ae60 con 0x7f0b5c0d4d60 2024-09-12T01:33:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.251+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1789759176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b5c0d4d60 msgr2=0x7f0b5c0d4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.251+0000 7f0b61bcf640 1 --2- 172.21.15.16:0/1789759176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b5c0d4d60 0x7f0b5c0d4640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0d21a0 tx=0x7f0b5002a110 comp rx=0 tx=0).stop 2024-09-12T01:33:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.251+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1789759176 shutdown_connections 2024-09-12T01:33:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.251+0000 7f0b61bcf640 1 --2- 172.21.15.16:0/1789759176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b5c0d4d60 0x7f0b5c0d4640 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.251+0000 7f0b61bcf640 1 --2- 172.21.15.16:0/1789759176 >> v2:172.21.15.33:3300/0 conn(0x7f0b5c15e660 0x7f0b5c0d4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.251+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1789759176 >> 172.21.15.16:0/1789759176 conn(0x7f0b5c08de10 msgr2=0x7f0b5c09d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.251+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1789759176 shutdown_connections 2024-09-12T01:33:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.251+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1789759176 wait complete. 2024-09-12T01:33:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.252+0000 7f0b61bcf640 1 Processor -- start 2024-09-12T01:33:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.253+0000 7f0b61bcf640 1 -- start start 2024-09-12T01:33:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.253+0000 7f0b61bcf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b5c0d4d60 0x7f0b5c161550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.253+0000 7f0b61bcf640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b5c15e660 0x7f0b5c0ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.253+0000 7f0b61bcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b5c162020 0x7f0b5c160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.253+0000 7f0b61bcf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c0d0fb0 con 0x7f0b5c162020 2024-09-12T01:33:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.253+0000 7f0b61bcf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c0d0e30 con 0x7f0b5c0d4d60 2024-09-12T01:33:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.253+0000 7f0b61bcf640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0b5c0d1130 con 0x7f0b5c15e660 2024-09-12T01:33:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.253+0000 7f0b5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b5c162020 0x7f0b5c160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.253+0000 7f0b5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b5c162020 0x7f0b5c160140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59172/0 (socket says 172.21.15.16:59172) 2024-09-12T01:33:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.253+0000 7f0b5bfff640 1 -- 172.21.15.16:0/1071416853 learned_addr learned my addr 172.21.15.16:0/1071416853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.254+0000 7f0b5b7fe640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b5c0d4d60 0x7f0b5c161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.254+0000 7f0b5affd640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b5c15e660 0x7f0b5c0ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.254+0000 7f0b5bfff640 1 -- 172.21.15.16:0/1071416853 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b5c15e660 msgr2=0x7f0b5c0ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.254+0000 7f0b5bfff640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b5c15e660 0x7f0b5c0ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.254+0000 7f0b5bfff640 1 -- 172.21.15.16:0/1071416853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b5c0d4d60 msgr2=0x7f0b5c161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.254+0000 7f0b5bfff640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b5c0d4d60 0x7f0b5c161550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.254+0000 7f0b5bfff640 1 -- 172.21.15.16:0/1071416853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5c0c9360 con 0x7f0b5c162020 2024-09-12T01:33:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.254+0000 7f0b5bfff640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b5c162020 0x7f0b5c160140 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400c7b0 tx=0x7f0b5400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.254+0000 7f0b43fff640 1 -- 172.21.15.16:0/1071416853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b54018070 con 0x7f0b5c162020 2024-09-12T01:33:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.254+0000 7f0b43fff640 1 -- 172.21.15.16:0/1071416853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5400a640 con 0x7f0b5c162020 2024-09-12T01:33:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.254+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1071416853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5c0c8ac0 con 0x7f0b5c162020 2024-09-12T01:33:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.254+0000 7f0b43fff640 1 -- 172.21.15.16:0/1071416853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5400ce60 con 0x7f0b5c162020 2024-09-12T01:33:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.254+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1071416853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b5c0c9080 con 0x7f0b5c162020 2024-09-12T01:33:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.255+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1071416853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b5c15aa10 con 0x7f0b5c162020 2024-09-12T01:33:22.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.256+0000 7f0b43fff640 1 -- 172.21.15.16:0/1071416853 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b5401b070 con 0x7f0b5c162020 2024-09-12T01:33:22.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.256+0000 7f0b43fff640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0b4806be30 0x7f0b4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:22.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.256+0000 7f0b43fff640 1 -- 172.21.15.16:0/1071416853 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0b5408a710 con 0x7f0b5c162020 2024-09-12T01:33:22.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.256+0000 7f0b5b7fe640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0b4806be30 0x7f0b4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:22.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.257+0000 7f0b5b7fe640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0b4806be30 0x7f0b4806e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c156e50 tx=0x7f0b4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.260+0000 7f0b43fff640 1 -- 172.21.15.16:0/1071416853 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0b54058e10 con 0x7f0b5c162020 2024-09-12T01:33:22.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.446+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1071416853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0b5c0cef60 con 0x7f0b5c162020 2024-09-12T01:33:22.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.447+0000 7f0b43fff640 1 -- 172.21.15.16:0/1071416853 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f0b54058710 con 0x7f0b5c162020 2024-09-12T01:33:22.446 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-12T01:33:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.448+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1071416853 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0b4806be30 msgr2=0x7f0b4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.448+0000 7f0b61bcf640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0b4806be30 0x7f0b4806e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c156e50 tx=0x7f0b4c008040 comp rx=0 tx=0).stop 2024-09-12T01:33:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.448+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1071416853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b5c162020 msgr2=0x7f0b5c160140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.448+0000 7f0b61bcf640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b5c162020 0x7f0b5c160140 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400c7b0 tx=0x7f0b5400cc70 comp rx=0 tx=0).stop 2024-09-12T01:33:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.449+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1071416853 shutdown_connections 2024-09-12T01:33:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.449+0000 7f0b61bcf640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0b4806be30 0x7f0b4806e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.449+0000 7f0b61bcf640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b5c162020 0x7f0b5c160140 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.449+0000 7f0b61bcf640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b5c15e660 0x7f0b5c0ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.449+0000 7f0b61bcf640 1 --2- 172.21.15.16:0/1071416853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b5c0d4d60 0x7f0b5c161550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.449+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1071416853 >> 172.21.15.16:0/1071416853 conn(0x7f0b5c08de10 msgr2=0x7f0b5c0cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.449+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1071416853 shutdown_connections 2024-09-12T01:33:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:22.449+0000 7f0b61bcf640 1 -- 172.21.15.16:0/1071416853 wait complete. 2024-09-12T01:33:22.459 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-12T01:33:22.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:22.459 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:33:22.483 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[41984]: fuse finished with error 0 and tester_r 0 2024-09-12T01:33:22.485 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:33:28.586 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:33:28.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:28.587 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-12T01:33:28.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:33:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.776+0000 7fb16f00e640 1 Processor -- start 2024-09-12T01:33:28.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.776+0000 7fb16f00e640 1 -- start start 2024-09-12T01:33:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16f00e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb16809aa30 0x7fb1680d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16f00e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb16809ae10 0x7fb16806cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16f00e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb168161ef0 0x7fb168162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16f00e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb16815e7f0 con 0x7fb168161ef0 2024-09-12T01:33:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16f00e640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb1680d1a50 con 0x7fb16809ae10 2024-09-12T01:33:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16f00e640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb1680d09b0 con 0x7fb16809aa30 2024-09-12T01:33:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16d5ab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb168161ef0 0x7fb168162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16d5ab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb168161ef0 0x7fb168162360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41946/0 (socket says 172.21.15.16:41946) 2024-09-12T01:33:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16d5ab640 1 -- 172.21.15.16:0/3739587998 learned_addr learned my addr 172.21.15.16:0/3739587998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb167fff640 1 --2- 172.21.15.16:0/3739587998 >> v2:172.21.15.33:3300/0 conn(0x7fb16809ae10 0x7fb16806cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16cdaa640 1 --2- 172.21.15.16:0/3739587998 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb16809aa30 0x7fb1680d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16d5ab640 1 -- 172.21.15.16:0/3739587998 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb16809aa30 msgr2=0x7fb1680d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16d5ab640 1 --2- 172.21.15.16:0/3739587998 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb16809aa30 0x7fb1680d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16d5ab640 1 -- 172.21.15.16:0/3739587998 >> v2:172.21.15.33:3300/0 conn(0x7fb16809ae10 msgr2=0x7fb16806cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16d5ab640 1 --2- 172.21.15.16:0/3739587998 >> v2:172.21.15.33:3300/0 conn(0x7fb16809ae10 0x7fb16806cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16d5ab640 1 -- 172.21.15.16:0/3739587998 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1680d0c90 con 0x7fb168161ef0 2024-09-12T01:33:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.777+0000 7fb16cdaa640 1 --2- 172.21.15.16:0/3739587998 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb16809aa30 0x7fb1680d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.778+0000 7fb16d5ab640 1 --2- 172.21.15.16:0/3739587998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb168161ef0 0x7fb168162360 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb168073660 tx=0x7fb16002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c7ddeadc57cd23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.778+0000 7fb1677fe640 1 -- 172.21.15.16:0/3739587998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160036070 con 0x7fb168161ef0 2024-09-12T01:33:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.778+0000 7fb1677fe640 1 -- 172.21.15.16:0/3739587998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb160031600 con 0x7fb168161ef0 2024-09-12T01:33:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.778+0000 7fb1677fe640 1 -- 172.21.15.16:0/3739587998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1600319d0 con 0x7fb168161ef0 2024-09-12T01:33:28.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.779+0000 7fb16f00e640 1 -- 172.21.15.16:0/3739587998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb168161ef0 msgr2=0x7fb168162360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.779+0000 7fb16f00e640 1 --2- 172.21.15.16:0/3739587998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb168161ef0 0x7fb168162360 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb168073660 tx=0x7fb16002a6a0 comp rx=0 tx=0).stop 2024-09-12T01:33:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.779+0000 7fb16f00e640 1 -- 172.21.15.16:0/3739587998 shutdown_connections 2024-09-12T01:33:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.779+0000 7fb16f00e640 1 --2- 172.21.15.16:0/3739587998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb168161ef0 0x7fb168162360 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.779+0000 7fb16f00e640 1 --2- 172.21.15.16:0/3739587998 >> v2:172.21.15.33:3300/0 conn(0x7fb16809ae10 0x7fb16806cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.779+0000 7fb16f00e640 1 --2- 172.21.15.16:0/3739587998 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb16809aa30 0x7fb1680d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.779+0000 7fb16f00e640 1 -- 172.21.15.16:0/3739587998 >> 172.21.15.16:0/3739587998 conn(0x7fb1680b3cf0 msgr2=0x7fb1680b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.779+0000 7fb16f00e640 1 -- 172.21.15.16:0/3739587998 shutdown_connections 2024-09-12T01:33:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.779+0000 7fb16f00e640 1 -- 172.21.15.16:0/3739587998 wait complete. 2024-09-12T01:33:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.780+0000 7fb16f00e640 1 Processor -- start 2024-09-12T01:33:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.780+0000 7fb16f00e640 1 -- start start 2024-09-12T01:33:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16f00e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16809aa30 0x7fb1681f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16f00e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb16809ae10 0x7fb16815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16f00e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb168161ef0 0x7fb1680b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16f00e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1680d1530 con 0x7fb168161ef0 2024-09-12T01:33:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16f00e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb1680d1250 con 0x7fb16809aa30 2024-09-12T01:33:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16f00e640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb16809b250 con 0x7fb16809ae10 2024-09-12T01:33:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16d5ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb168161ef0 0x7fb1680b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16d5ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb168161ef0 0x7fb1680b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41954/0 (socket says 172.21.15.16:41954) 2024-09-12T01:33:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16d5ab640 1 -- 172.21.15.16:0/1056619297 learned_addr learned my addr 172.21.15.16:0/1056619297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb167fff640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb16809ae10 0x7fb16815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16cdaa640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16809aa30 0x7fb1681f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16d5ab640 1 -- 172.21.15.16:0/1056619297 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb16809ae10 msgr2=0x7fb16815d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16d5ab640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb16809ae10 0x7fb16815d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16d5ab640 1 -- 172.21.15.16:0/1056619297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16809aa30 msgr2=0x7fb1681f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16d5ab640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16809aa30 0x7fb1681f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.781+0000 7fb16d5ab640 1 -- 172.21.15.16:0/1056619297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1680c71b0 con 0x7fb168161ef0 2024-09-12T01:33:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.782+0000 7fb16d5ab640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb168161ef0 0x7fb1680b73c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb16002ac90 tx=0x7fb160004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.782+0000 7fb1657fa640 1 -- 172.21.15.16:0/1056619297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16003f020 con 0x7fb168161ef0 2024-09-12T01:33:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.782+0000 7fb1657fa640 1 -- 172.21.15.16:0/1056619297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb160037040 con 0x7fb168161ef0 2024-09-12T01:33:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.782+0000 7fb16f00e640 1 -- 172.21.15.16:0/1056619297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1680c7490 con 0x7fb168161ef0 2024-09-12T01:33:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.782+0000 7fb1657fa640 1 -- 172.21.15.16:0/1056619297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160036070 con 0x7fb168161ef0 2024-09-12T01:33:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.782+0000 7fb16f00e640 1 -- 172.21.15.16:0/1056619297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1680c7a50 con 0x7fb168161ef0 2024-09-12T01:33:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.783+0000 7fb16f00e640 1 -- 172.21.15.16:0/1056619297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1680d0350 con 0x7fb168161ef0 2024-09-12T01:33:28.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.783+0000 7fb1657fa640 1 -- 172.21.15.16:0/1056619297 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb160042020 con 0x7fb168161ef0 2024-09-12T01:33:28.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.784+0000 7fb1657fa640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb14c06be30 0x7fb14c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:28.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.784+0000 7fb1657fa640 1 -- 172.21.15.16:0/1056619297 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb1600ab510 con 0x7fb168161ef0 2024-09-12T01:33:28.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.784+0000 7fb16cdaa640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb14c06be30 0x7fb14c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:28.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.784+0000 7fb16cdaa640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb14c06be30 0x7fb14c06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00aac0 tx=0x7fb15c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:28.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.788+0000 7fb1657fa640 1 -- 172.21.15.16:0/1056619297 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb160049030 con 0x7fb168161ef0 2024-09-12T01:33:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.965+0000 7fb16f00e640 1 -- 172.21.15.16:0/1056619297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb168161a10 con 0x7fb168161ef0 2024-09-12T01:33:28.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.966+0000 7fb1657fa640 1 -- 172.21.15.16:0/1056619297 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18574 (secure 0 0 0) 0x7fb1600795c0 con 0x7fb168161ef0 2024-09-12T01:33:28.966 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:28.966 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:33:18.213622+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:33:09.364639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:33:10.478237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:33:14.204624+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:33:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.969+0000 7fb16f00e640 1 -- 172.21.15.16:0/1056619297 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb14c06be30 msgr2=0x7fb14c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.969+0000 7fb16f00e640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb14c06be30 0x7fb14c06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00aac0 tx=0x7fb15c008040 comp rx=0 tx=0).stop 2024-09-12T01:33:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.969+0000 7fb16f00e640 1 -- 172.21.15.16:0/1056619297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb168161ef0 msgr2=0x7fb1680b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.969+0000 7fb16f00e640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb168161ef0 0x7fb1680b73c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb16002ac90 tx=0x7fb160004a40 comp rx=0 tx=0).stop 2024-09-12T01:33:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.969+0000 7fb16f00e640 1 -- 172.21.15.16:0/1056619297 shutdown_connections 2024-09-12T01:33:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.969+0000 7fb16f00e640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb14c06be30 0x7fb14c06e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.969+0000 7fb16f00e640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb168161ef0 0x7fb1680b73c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.969+0000 7fb16f00e640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb16809ae10 0x7fb16815d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.969+0000 7fb16f00e640 1 --2- 172.21.15.16:0/1056619297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb16809aa30 0x7fb1681f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.969+0000 7fb16f00e640 1 -- 172.21.15.16:0/1056619297 >> 172.21.15.16:0/1056619297 conn(0x7fb1680b3cf0 msgr2=0x7fb168160980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.969+0000 7fb16f00e640 1 -- 172.21.15.16:0/1056619297 shutdown_connections 2024-09-12T01:33:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:28.969+0000 7fb16f00e640 1 -- 172.21.15.16:0/1056619297 wait complete. 2024-09-12T01:33:28.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:33:29.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.083+0000 7f827e54b640 1 Processor -- start 2024-09-12T01:33:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.083+0000 7f827e54b640 1 -- start start 2024-09-12T01:33:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.083+0000 7f827e54b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f827815ff00 0x7f82780cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.083+0000 7f827e54b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82781608e0 con 0x7f8278156940 2024-09-12T01:33:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.083+0000 7f827e54b640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8278160bc0 con 0x7f8278160390 2024-09-12T01:33:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.083+0000 7f827e54b640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f827815ce90 con 0x7f827815ff00 2024-09-12T01:33:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.084+0000 7f8277fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8278156940 0x7f827815d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53564/0 (socket says 172.21.15.16:53564) 2024-09-12T01:33:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.084+0000 7f8277fff640 1 -- 172.21.15.16:0/3288251926 learned_addr learned my addr 172.21.15.16:0/3288251926 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.084+0000 7f82777fe640 1 --2- 172.21.15.16:0/3288251926 >> v2:172.21.15.33:3300/0 conn(0x7f827815ff00 0x7f82780cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.084+0000 7f8276ffd640 1 -- 172.21.15.16:0/3288251926 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3760967549 0 0) 0x7f82781608e0 con 0x7f8278156940 2024-09-12T01:33:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.084+0000 7f8276ffd640 1 -- 172.21.15.16:0/3288251926 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8260003520 con 0x7f8278156940 2024-09-12T01:33:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.084+0000 7f8276ffd640 1 -- 172.21.15.16:0/3288251926 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3186638580 0 0) 0x7f8278160bc0 con 0x7f8278160390 2024-09-12T01:33:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.084+0000 7f8276ffd640 1 -- 172.21.15.16:0/3288251926 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82781608e0 con 0x7f8278160390 2024-09-12T01:33:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.084+0000 7f8276ffd640 1 -- 172.21.15.16:0/3288251926 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2682475352 0 0) 0x7f8260003520 con 0x7f8278156940 2024-09-12T01:33:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.084+0000 7f82777fe640 1 -- 172.21.15.16:0/3288251926 >> v1:172.21.15.33:6789/0 conn(0x7f8278160390 legacy=0x7f82780cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.084+0000 7f82777fe640 1 -- 172.21.15.16:0/3288251926 >> v1:172.21.15.16:6789/0 conn(0x7f8278156940 legacy=0x7f827815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.084+0000 7f82777fe640 1 -- 172.21.15.16:0/3288251926 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827815cb20 con 0x7f827815ff00 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.085+0000 7f82777fe640 1 --2- 172.21.15.16:0/3288251926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827815ff00 0x7f82780cc820 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f826c0097a0 tx=0x7f826c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82416b8820f1fe77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.085+0000 7f8276ffd640 1 -- 172.21.15.16:0/3288251926 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c037070 con 0x7f827815ff00 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.085+0000 7f8276ffd640 1 -- 172.21.15.16:0/3288251926 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826c02ac50 con 0x7f827815ff00 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.085+0000 7f8276ffd640 1 -- 172.21.15.16:0/3288251926 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268001670 con 0x7f827815ff00 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.086+0000 7f827e54b640 1 -- 172.21.15.16:0/3288251926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827815ff00 msgr2=0x7f82780cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.086+0000 7f827e54b640 1 --2- 172.21.15.16:0/3288251926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827815ff00 0x7f82780cc820 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f826c0097a0 tx=0x7f826c02a1c0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.086+0000 7f827e54b640 1 -- 172.21.15.16:0/3288251926 shutdown_connections 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.086+0000 7f827e54b640 1 --2- 172.21.15.16:0/3288251926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827815ff00 0x7f82780cc820 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.086+0000 7f827e54b640 1 -- 172.21.15.16:0/3288251926 >> 172.21.15.16:0/3288251926 conn(0x7f827806fdc0 msgr2=0x7f82780701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.086+0000 7f827e54b640 1 -- 172.21.15.16:0/3288251926 shutdown_connections 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.086+0000 7f827e54b640 1 -- 172.21.15.16:0/3288251926 wait complete. 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.087+0000 7f827e54b640 1 Processor -- start 2024-09-12T01:33:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.087+0000 7f827e54b640 1 -- start start 2024-09-12T01:33:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827e54b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8278156940 0x7f82781f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827e54b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827815ff00 0x7f82781f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827e54b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8278160390 0x7f82781f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827e54b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82780d1b30 con 0x7f8278160390 2024-09-12T01:33:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827e54b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82780d19b0 con 0x7f827815ff00 2024-09-12T01:33:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827e54b640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f82780ccd60 con 0x7f8278156940 2024-09-12T01:33:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827cae8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8278160390 0x7f82781f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827cae8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8278160390 0x7f82781f4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41956/0 (socket says 172.21.15.16:41956) 2024-09-12T01:33:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827cae8640 1 -- 172.21.15.16:0/1534229547 learned_addr learned my addr 172.21.15.16:0/1534229547 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f8277fff640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8278156940 0x7f82781f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f82777fe640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827815ff00 0x7f82781f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827cae8640 1 -- 172.21.15.16:0/1534229547 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8278156940 msgr2=0x7f82781f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827cae8640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8278156940 0x7f82781f3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827cae8640 1 -- 172.21.15.16:0/1534229547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827815ff00 msgr2=0x7f82781f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827cae8640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827815ff00 0x7f82781f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827cae8640 1 -- 172.21.15.16:0/1534229547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82781f5040 con 0x7f8278160390 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f8277fff640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8278156940 0x7f82781f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f82777fe640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827815ff00 0x7f82781f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.088+0000 7f827cae8640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8278160390 0x7f82781f4a40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f827000c6e0 tx=0x7f827000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.089+0000 7f8274ff9640 1 -- 172.21.15.16:0/1534229547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270003b70 con 0x7f8278160390 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.089+0000 7f8274ff9640 1 -- 172.21.15.16:0/1534229547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8270003d10 con 0x7f8278160390 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.089+0000 7f8274ff9640 1 -- 172.21.15.16:0/1534229547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82700052d0 con 0x7f8278160390 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.089+0000 7f827e54b640 1 -- 172.21.15.16:0/1534229547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827806d610 con 0x7f8278160390 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.089+0000 7f827e54b640 1 -- 172.21.15.16:0/1534229547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827806dbd0 con 0x7f8278160390 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.090+0000 7f827e54b640 1 -- 172.21.15.16:0/1534229547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827806ea90 con 0x7f8278160390 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.090+0000 7f8274ff9640 1 -- 172.21.15.16:0/1534229547 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8270005470 con 0x7f8278160390 2024-09-12T01:33:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.091+0000 7f8274ff9640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f826406be30 0x7f826406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.091+0000 7f8274ff9640 1 -- 172.21.15.16:0/1534229547 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f827008b150 con 0x7f8278160390 2024-09-12T01:33:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.091+0000 7f8277fff640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f826406be30 0x7f826406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.091+0000 7f8277fff640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f826406be30 0x7f826406e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f82680011e0 tx=0x7f8268009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.095+0000 7f8274ff9640 1 -- 172.21.15.16:0/1534229547 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8270059740 con 0x7f8278160390 2024-09-12T01:33:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.274+0000 7f827e54b640 1 -- 172.21.15.16:0/1534229547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82780cc820 con 0x7f8278160390 2024-09-12T01:33:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.275+0000 7f8274ff9640 1 -- 172.21.15.16:0/1534229547 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18574 (secure 0 0 0) 0x7f8270059040 con 0x7f8278160390 2024-09-12T01:33:29.276 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:29.276 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:33:18.213622+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:33:09.364639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:33:10.478237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:33:14.204624+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:33:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.278+0000 7f827e54b640 1 -- 172.21.15.16:0/1534229547 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f826406be30 msgr2=0x7f826406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.278+0000 7f827e54b640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f826406be30 0x7f826406e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f82680011e0 tx=0x7f8268009040 comp rx=0 tx=0).stop 2024-09-12T01:33:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.278+0000 7f827e54b640 1 -- 172.21.15.16:0/1534229547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8278160390 msgr2=0x7f82781f4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.278+0000 7f827e54b640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8278160390 0x7f82781f4a40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f827000c6e0 tx=0x7f827000cba0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.278+0000 7f827e54b640 1 -- 172.21.15.16:0/1534229547 shutdown_connections 2024-09-12T01:33:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.278+0000 7f827e54b640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f826406be30 0x7f826406e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.278+0000 7f827e54b640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8278160390 0x7f82781f4a40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.278+0000 7f827e54b640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827815ff00 0x7f82781f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.278+0000 7f827e54b640 1 --2- 172.21.15.16:0/1534229547 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8278156940 0x7f82781f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.278+0000 7f827e54b640 1 -- 172.21.15.16:0/1534229547 >> 172.21.15.16:0/1534229547 conn(0x7f827806fdc0 msgr2=0x7f82780d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.278+0000 7f827e54b640 1 -- 172.21.15.16:0/1534229547 shutdown_connections 2024-09-12T01:33:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.278+0000 7f827e54b640 1 -- 172.21.15.16:0/1534229547 wait complete. 2024-09-12T01:33:29.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T01:33:29.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.408+0000 7f1dc30ba640 1 Processor -- start 2024-09-12T01:33:29.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.408+0000 7f1dc30ba640 1 -- start start 2024-09-12T01:33:29.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.409+0000 7f1dc30ba640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1dbc0d1540 0x7f1dbc0d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:29.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.409+0000 7f1dc30ba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dbc0cf050 con 0x7f1dbc15e6a0 2024-09-12T01:33:29.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.409+0000 7f1dc30ba640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dbc0cf2d0 con 0x7f1dbc0d0090 2024-09-12T01:33:29.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.409+0000 7f1dc30ba640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f1dbc0cfc00 con 0x7f1dbc0d1540 2024-09-12T01:33:29.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.409+0000 7f1dc0e56640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1dbc15e6a0 0x7f1dbc0d4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53568/0 (socket says 172.21.15.16:53568) 2024-09-12T01:33:29.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.409+0000 7f1dc0e56640 1 -- 172.21.15.16:0/660879508 learned_addr learned my addr 172.21.15.16:0/660879508 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:29.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.409+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3738239696 0 0) 0x7f1dac001290 con 0x7f1dbc15e6a0 2024-09-12T01:33:29.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1da4003520 con 0x7f1dbc15e6a0 2024-09-12T01:33:29.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dc1657640 1 --2- 172.21.15.16:0/660879508 >> v2:172.21.15.33:3300/0 conn(0x7f1dbc0d1540 0x7f1dbc0d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3833887772 0 0) 0x7f1da4003520 con 0x7f1dbc15e6a0 2024-09-12T01:33:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 >> v2:172.21.15.33:3300/0 conn(0x7f1dbc0d1540 msgr2=0x7f1dbc0d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 --2- 172.21.15.16:0/660879508 >> v2:172.21.15.33:3300/0 conn(0x7f1dbc0d1540 0x7f1dbc0d3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 >> v1:172.21.15.33:6789/0 conn(0x7f1dbc0d0090 legacy=0x7f1dbc0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc0cf5b0 con 0x7f1dbc15e6a0 2024-09-12T01:33:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f1dac0019c0 con 0x7f1dbc15e6a0 2024-09-12T01:33:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 >> v1:172.21.15.16:6789/0 conn(0x7f1dbc15e6a0 legacy=0x7f1dbc0d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 --2- 172.21.15.16:0/660879508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da40039b0 0x7f1da4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 --2- 172.21.15.16:0/660879508 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1da4004800 0x7f1da4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:29.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 --2- 172.21.15.16:0/660879508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1da40051c0 0x7f1da400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:29.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc0cfc00 con 0x7f1da40051c0 2024-09-12T01:33:29.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1da400c110 con 0x7f1da40039b0 2024-09-12T01:33:29.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f1da400c310 con 0x7f1da4004800 2024-09-12T01:33:29.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dc1657640 1 --2- 172.21.15.16:0/660879508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1da40051c0 0x7f1da400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:29.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dc0e56640 1 --2- 172.21.15.16:0/660879508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da40039b0 0x7f1da4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:29.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dbbfff640 1 --2- 172.21.15.16:0/660879508 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1da4004800 0x7f1da4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:29.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dc1657640 1 -- 172.21.15.16:0/660879508 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1da4004800 msgr2=0x7f1da4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:29.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dc1657640 1 --2- 172.21.15.16:0/660879508 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1da4004800 0x7f1da4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dc1657640 1 -- 172.21.15.16:0/660879508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da40039b0 msgr2=0x7f1da4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:29.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dc1657640 1 --2- 172.21.15.16:0/660879508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da40039b0 0x7f1da4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.410+0000 7f1dc1657640 1 -- 172.21.15.16:0/660879508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da4003520 con 0x7f1da40051c0 2024-09-12T01:33:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.411+0000 7f1dc1657640 1 -- 172.21.15.16:0/660879508 reap_dead start 2024-09-12T01:33:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.411+0000 7f1dc1657640 1 --2- 172.21.15.16:0/660879508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1da40051c0 0x7f1da400b9c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1db4007360 tx=0x7f1da4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcef7d2c6a76bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.411+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4009be0 con 0x7f1da40051c0 2024-09-12T01:33:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.411+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db4003bd0 con 0x7f1da40051c0 2024-09-12T01:33:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.411+0000 7f1dbb7fe640 1 -- 172.21.15.16:0/660879508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4004020 con 0x7f1da40051c0 2024-09-12T01:33:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.412+0000 7f1dc30ba640 1 -- 172.21.15.16:0/660879508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1da40051c0 msgr2=0x7f1da400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.412+0000 7f1dc30ba640 1 --2- 172.21.15.16:0/660879508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1da40051c0 0x7f1da400b9c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1db4007360 tx=0x7f1da4005830 comp rx=0 tx=0).stop 2024-09-12T01:33:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.412+0000 7f1dc30ba640 1 -- 172.21.15.16:0/660879508 shutdown_connections 2024-09-12T01:33:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.412+0000 7f1dc30ba640 1 --2- 172.21.15.16:0/660879508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1da40051c0 0x7f1da400b9c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.412+0000 7f1dc30ba640 1 -- 172.21.15.16:0/660879508 >> 172.21.15.16:0/660879508 conn(0x7f1dbc09d730 msgr2=0x7f1dbc0d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.412+0000 7f1dc30ba640 1 -- 172.21.15.16:0/660879508 shutdown_connections 2024-09-12T01:33:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.412+0000 7f1dc30ba640 1 -- 172.21.15.16:0/660879508 wait complete. 2024-09-12T01:33:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.413+0000 7f1dc30ba640 1 Processor -- start 2024-09-12T01:33:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.413+0000 7f1dc30ba640 1 -- start start 2024-09-12T01:33:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dc30ba640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1da40051c0 0x7f1dbc0c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dc30ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dbc154850 0x7f1dbc15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dc30ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dbc15d820 0x7f1dbc0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dc30ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc0d32e0 con 0x7f1dbc154850 2024-09-12T01:33:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dc30ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc0d3160 con 0x7f1dbc15d820 2024-09-12T01:33:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dc30ba640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f1dbc0d3460 con 0x7f1da40051c0 2024-09-12T01:33:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dbbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dbc154850 0x7f1dbc15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dbbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dbc154850 0x7f1dbc15d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41972/0 (socket says 172.21.15.16:41972) 2024-09-12T01:33:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dbbfff640 1 -- 172.21.15.16:0/2283257012 learned_addr learned my addr 172.21.15.16:0/2283257012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dc0e56640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1da40051c0 0x7f1dbc0c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dc1657640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dbc15d820 0x7f1dbc0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dbbfff640 1 -- 172.21.15.16:0/2283257012 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1da40051c0 msgr2=0x7f1dbc0c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dbbfff640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1da40051c0 0x7f1dbc0c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dbbfff640 1 -- 172.21.15.16:0/2283257012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dbc15d820 msgr2=0x7f1dbc0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dbbfff640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dbc15d820 0x7f1dbc0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dbbfff640 1 -- 172.21.15.16:0/2283257012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc0c82f0 con 0x7f1dbc154850 2024-09-12T01:33:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dc0e56640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1da40051c0 0x7f1dbc0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.414+0000 7f1dc1657640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dbc15d820 0x7f1dbc0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.415+0000 7f1dbbfff640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dbc154850 0x7f1dbc15d2e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1db000e410 tx=0x7f1db000e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.415+0000 7f1db97fa640 1 -- 172.21.15.16:0/2283257012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0019070 con 0x7f1dbc154850 2024-09-12T01:33:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.415+0000 7f1db97fa640 1 -- 172.21.15.16:0/2283257012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db00052e0 con 0x7f1dbc154850 2024-09-12T01:33:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.415+0000 7f1dc30ba640 1 -- 172.21.15.16:0/2283257012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dbc0c85d0 con 0x7f1dbc154850 2024-09-12T01:33:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.415+0000 7f1db97fa640 1 -- 172.21.15.16:0/2283257012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db00037f0 con 0x7f1dbc154850 2024-09-12T01:33:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.415+0000 7f1dc30ba640 1 -- 172.21.15.16:0/2283257012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dbc0c8b90 con 0x7f1dbc154850 2024-09-12T01:33:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.416+0000 7f1dc30ba640 1 -- 172.21.15.16:0/2283257012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dbc0a0e20 con 0x7f1dbc154850 2024-09-12T01:33:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.416+0000 7f1db97fa640 1 -- 172.21.15.16:0/2283257012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1db001c070 con 0x7f1dbc154850 2024-09-12T01:33:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.417+0000 7f1db97fa640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1da806be30 0x7f1da806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.417+0000 7f1db97fa640 1 -- 172.21.15.16:0/2283257012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1db008a740 con 0x7f1dbc154850 2024-09-12T01:33:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.417+0000 7f1dc0e56640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1da806be30 0x7f1da806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.417+0000 7f1dc0e56640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1da806be30 0x7f1da806e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1dac000ef0 tx=0x7f1dac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.421+0000 7f1db97fa640 1 -- 172.21.15.16:0/2283257012 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1db0058760 con 0x7f1dbc154850 2024-09-12T01:33:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:29.635+0000 7f1dc30ba640 1 -- 172.21.15.16:0/2283257012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1dbc06df80 con 0x7f1dbc154850 2024-09-12T01:33:30.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.249+0000 7f1db97fa640 1 -- 172.21.15.16:0/2283257012 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f1db005ca60 con 0x7f1dbc154850 2024-09-12T01:33:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.252+0000 7f1dc30ba640 1 -- 172.21.15.16:0/2283257012 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1da806be30 msgr2=0x7f1da806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.252+0000 7f1dc30ba640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1da806be30 0x7f1da806e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1dac000ef0 tx=0x7f1dac009040 comp rx=0 tx=0).stop 2024-09-12T01:33:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.252+0000 7f1dc30ba640 1 -- 172.21.15.16:0/2283257012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dbc154850 msgr2=0x7f1dbc15d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.252+0000 7f1dc30ba640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dbc154850 0x7f1dbc15d2e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1db000e410 tx=0x7f1db000e8d0 comp rx=0 tx=0).stop 2024-09-12T01:33:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.252+0000 7f1dc30ba640 1 -- 172.21.15.16:0/2283257012 shutdown_connections 2024-09-12T01:33:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.252+0000 7f1dc30ba640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1da806be30 0x7f1da806e2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:30.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.252+0000 7f1dc30ba640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dbc15d820 0x7f1dbc0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:30.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.252+0000 7f1dc30ba640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dbc154850 0x7f1dbc15d2e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:30.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.252+0000 7f1dc30ba640 1 --2- 172.21.15.16:0/2283257012 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1da40051c0 0x7f1dbc0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:30.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.252+0000 7f1dc30ba640 1 -- 172.21.15.16:0/2283257012 >> 172.21.15.16:0/2283257012 conn(0x7f1dbc09d730 msgr2=0x7f1dbc160060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:30.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.252+0000 7f1dc30ba640 1 -- 172.21.15.16:0/2283257012 shutdown_connections 2024-09-12T01:33:30.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.252+0000 7f1dc30ba640 1 -- 172.21.15.16:0/2283257012 wait complete. 2024-09-12T01:33:30.266 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-12T01:33:30.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T01:33:30.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.379+0000 7fb33d465640 1 Processor -- start 2024-09-12T01:33:30.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.379+0000 7fb33d465640 1 -- start start 2024-09-12T01:33:30.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.380+0000 7fb33d465640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb33815e6a0 0x7fb3380d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:30.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.380+0000 7fb33d465640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb3380d3160 0x7fb3380d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:30.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.380+0000 7fb33d465640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3380cf050 con 0x7fb3380d0e30 2024-09-12T01:33:30.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.380+0000 7fb33d465640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb3380cfc00 con 0x7fb33815e6a0 2024-09-12T01:33:30.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.380+0000 7fb33d465640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb3380cf2d0 con 0x7fb3380d3160 2024-09-12T01:33:30.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.380+0000 7fb336ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb33815e6a0 0x7fb3380d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:30.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.380+0000 7fb336ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb33815e6a0 0x7fb3380d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41976/0 (socket says 172.21.15.16:41976) 2024-09-12T01:33:30.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.380+0000 7fb336ffd640 1 -- 172.21.15.16:0/1183544119 learned_addr learned my addr 172.21.15.16:0/1183544119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:30.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.380+0000 7fb32ffff640 1 --2- 172.21.15.16:0/1183544119 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb3380d3160 0x7fb3380d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:30.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.380+0000 7fb336ffd640 1 -- 172.21.15.16:0/1183544119 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb3380d3160 msgr2=0x7fb3380d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:30.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.380+0000 7fb336ffd640 1 --2- 172.21.15.16:0/1183544119 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb3380d3160 0x7fb3380d4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:30.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.380+0000 7fb336ffd640 1 -- 172.21.15.16:0/1183544119 >> v1:172.21.15.33:6789/0 conn(0x7fb3380d0e30 legacy=0x7fb3380b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:30.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.380+0000 7fb336ffd640 1 -- 172.21.15.16:0/1183544119 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3380cf5b0 con 0x7fb33815e6a0 2024-09-12T01:33:30.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.381+0000 7fb336ffd640 1 --2- 172.21.15.16:0/1183544119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33815e6a0 0x7fb3380d0090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb328009850 tx=0x7fb32802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb91b8ec49c761e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:30.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.381+0000 7fb3367fc640 1 -- 172.21.15.16:0/1183544119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328004030 con 0x7fb33815e6a0 2024-09-12T01:33:30.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.381+0000 7fb3367fc640 1 -- 172.21.15.16:0/1183544119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3280041d0 con 0x7fb33815e6a0 2024-09-12T01:33:30.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.381+0000 7fb3367fc640 1 -- 172.21.15.16:0/1183544119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328003610 con 0x7fb33815e6a0 2024-09-12T01:33:30.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.382+0000 7fb33d465640 1 -- 172.21.15.16:0/1183544119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33815e6a0 msgr2=0x7fb3380d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:30.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.382+0000 7fb33d465640 1 --2- 172.21.15.16:0/1183544119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33815e6a0 0x7fb3380d0090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb328009850 tx=0x7fb32802a1e0 comp rx=0 tx=0).stop 2024-09-12T01:33:30.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.382+0000 7fb33d465640 1 -- 172.21.15.16:0/1183544119 shutdown_connections 2024-09-12T01:33:30.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.382+0000 7fb33d465640 1 --2- 172.21.15.16:0/1183544119 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb3380d3160 0x7fb3380d4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:30.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.382+0000 7fb33d465640 1 --2- 172.21.15.16:0/1183544119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33815e6a0 0x7fb3380d0090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:30.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.382+0000 7fb33d465640 1 -- 172.21.15.16:0/1183544119 >> 172.21.15.16:0/1183544119 conn(0x7fb33809d730 msgr2=0x7fb3380d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:30.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.382+0000 7fb33d465640 1 -- 172.21.15.16:0/1183544119 shutdown_connections 2024-09-12T01:33:30.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.382+0000 7fb33d465640 1 -- 172.21.15.16:0/1183544119 wait complete. 2024-09-12T01:33:30.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.383+0000 7fb33d465640 1 Processor -- start 2024-09-12T01:33:30.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.383+0000 7fb33d465640 1 -- start start 2024-09-12T01:33:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb33d465640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3380d0e30 0x7fb338157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb33d465640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb3380d3160 0x7fb33815ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb33d465640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33815e6a0 0x7fb338158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb33d465640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3380d51f0 con 0x7fb33815e6a0 2024-09-12T01:33:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb33d465640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb3380d5070 con 0x7fb3380d0e30 2024-09-12T01:33:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb33d465640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb3380d12e0 con 0x7fb3380d3160 2024-09-12T01:33:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb3377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33815e6a0 0x7fb338158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb3377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33815e6a0 0x7fb338158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41980/0 (socket says 172.21.15.16:41980) 2024-09-12T01:33:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb3377fe640 1 -- 172.21.15.16:0/3989640448 learned_addr learned my addr 172.21.15.16:0/3989640448 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb32ffff640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb3380d3160 0x7fb33815ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb336ffd640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3380d0e30 0x7fb338157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb3377fe640 1 -- 172.21.15.16:0/3989640448 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb3380d3160 msgr2=0x7fb33815ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb3377fe640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb3380d3160 0x7fb33815ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb3377fe640 1 -- 172.21.15.16:0/3989640448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3380d0e30 msgr2=0x7fb338157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb3377fe640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3380d0e30 0x7fb338157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.384+0000 7fb3377fe640 1 -- 172.21.15.16:0/3989640448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3380c82f0 con 0x7fb33815e6a0 2024-09-12T01:33:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.385+0000 7fb3377fe640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33815e6a0 0x7fb338158380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb33000c7b0 tx=0x7fb33000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.385+0000 7fb32f7fe640 1 -- 172.21.15.16:0/3989640448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb330004b60 con 0x7fb33815e6a0 2024-09-12T01:33:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.385+0000 7fb32f7fe640 1 -- 172.21.15.16:0/3989640448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb330004d00 con 0x7fb33815e6a0 2024-09-12T01:33:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.385+0000 7fb33d465640 1 -- 172.21.15.16:0/3989640448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3380c85d0 con 0x7fb33815e6a0 2024-09-12T01:33:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.385+0000 7fb32f7fe640 1 -- 172.21.15.16:0/3989640448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33000aa90 con 0x7fb33815e6a0 2024-09-12T01:33:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.385+0000 7fb33d465640 1 -- 172.21.15.16:0/3989640448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3380c8b90 con 0x7fb33815e6a0 2024-09-12T01:33:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.386+0000 7fb33d465640 1 -- 172.21.15.16:0/3989640448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3380d1660 con 0x7fb33815e6a0 2024-09-12T01:33:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.386+0000 7fb32f7fe640 1 -- 172.21.15.16:0/3989640448 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb33000ac30 con 0x7fb33815e6a0 2024-09-12T01:33:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.387+0000 7fb32f7fe640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb31806be30 0x7fb31806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.387+0000 7fb32f7fe640 1 -- 172.21.15.16:0/3989640448 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb33008b2f0 con 0x7fb33815e6a0 2024-09-12T01:33:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.387+0000 7fb336ffd640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb31806be30 0x7fb31806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.387+0000 7fb336ffd640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb31806be30 0x7fb31806e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb328033000 tx=0x7fb32802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:30.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.391+0000 7fb32f7fe640 1 -- 172.21.15.16:0/3989640448 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb330020070 con 0x7fb33815e6a0 2024-09-12T01:33:30.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:30.601+0000 7fb33d465640 1 -- 172.21.15.16:0/3989640448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb33806d800 con 0x7fb33815e6a0 2024-09-12T01:33:31.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.257+0000 7fb32f7fe640 1 -- 172.21.15.16:0/3989640448 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fb33008b110 con 0x7fb33815e6a0 2024-09-12T01:33:31.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.260+0000 7fb33d465640 1 -- 172.21.15.16:0/3989640448 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb31806be30 msgr2=0x7fb31806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.260+0000 7fb33d465640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb31806be30 0x7fb31806e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb328033000 tx=0x7fb32802e000 comp rx=0 tx=0).stop 2024-09-12T01:33:31.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.260+0000 7fb33d465640 1 -- 172.21.15.16:0/3989640448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33815e6a0 msgr2=0x7fb338158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.260+0000 7fb33d465640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33815e6a0 0x7fb338158380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb33000c7b0 tx=0x7fb33000cc70 comp rx=0 tx=0).stop 2024-09-12T01:33:31.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.260+0000 7fb33d465640 1 -- 172.21.15.16:0/3989640448 shutdown_connections 2024-09-12T01:33:31.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.260+0000 7fb33d465640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb31806be30 0x7fb31806e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.260+0000 7fb33d465640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33815e6a0 0x7fb338158380 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.260+0000 7fb33d465640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb3380d3160 0x7fb33815ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.260+0000 7fb33d465640 1 --2- 172.21.15.16:0/3989640448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3380d0e30 0x7fb338157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.260+0000 7fb33d465640 1 -- 172.21.15.16:0/3989640448 >> 172.21.15.16:0/3989640448 conn(0x7fb33809d730 msgr2=0x7fb33815f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:31.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.260+0000 7fb33d465640 1 -- 172.21.15.16:0/3989640448 shutdown_connections 2024-09-12T01:33:31.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.260+0000 7fb33d465640 1 -- 172.21.15.16:0/3989640448 wait complete. 2024-09-12T01:33:31.273 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T01:33:31.273 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:33:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.384+0000 7ff90c027640 1 Processor -- start 2024-09-12T01:33:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.385+0000 7ff90c027640 1 -- start start 2024-09-12T01:33:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.385+0000 7ff90c027640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff904156940 0x7ff90415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.385+0000 7ff90c027640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff90415ff00 0x7ff9040cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.385+0000 7ff90c027640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9041608e0 con 0x7ff904160390 2024-09-12T01:33:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.385+0000 7ff90c027640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff90415ce90 con 0x7ff90415ff00 2024-09-12T01:33:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.385+0000 7ff90c027640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7ff904160bc0 con 0x7ff904156940 2024-09-12T01:33:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.386+0000 7ff9095c2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff90415ff00 0x7ff9040cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.386+0000 7ff9095c2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff90415ff00 0x7ff9040cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41988/0 (socket says 172.21.15.16:41988) 2024-09-12T01:33:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.386+0000 7ff9095c2640 1 -- 172.21.15.16:0/4112148363 learned_addr learned my addr 172.21.15.16:0/4112148363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.386+0000 7ff909dc3640 1 --2- 172.21.15.16:0/4112148363 >> v2:172.21.15.33:3300/0 conn(0x7ff904156940 0x7ff90415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.386+0000 7ff9095c2640 1 -- 172.21.15.16:0/4112148363 >> v2:172.21.15.33:3300/0 conn(0x7ff904156940 msgr2=0x7ff90415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.386+0000 7ff9095c2640 1 --2- 172.21.15.16:0/4112148363 >> v2:172.21.15.33:3300/0 conn(0x7ff904156940 0x7ff90415d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.386+0000 7ff9095c2640 1 -- 172.21.15.16:0/4112148363 >> v1:172.21.15.33:6789/0 conn(0x7ff904160390 legacy=0x7ff9040cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.386+0000 7ff9095c2640 1 -- 172.21.15.16:0/4112148363 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90415cb20 con 0x7ff90415ff00 2024-09-12T01:33:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.387+0000 7ff9095c2640 1 --2- 172.21.15.16:0/4112148363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90415ff00 0x7ff9040cc820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff904073300 tx=0x7ff8f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e78a52e8b693aaf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.387+0000 7ff908dc1640 1 -- 172.21.15.16:0/4112148363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4037070 con 0x7ff90415ff00 2024-09-12T01:33:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.387+0000 7ff908dc1640 1 -- 172.21.15.16:0/4112148363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f4032440 con 0x7ff90415ff00 2024-09-12T01:33:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.387+0000 7ff908dc1640 1 -- 172.21.15.16:0/4112148363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4032810 con 0x7ff90415ff00 2024-09-12T01:33:31.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.388+0000 7ff90c027640 1 -- 172.21.15.16:0/4112148363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90415ff00 msgr2=0x7ff9040cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.388+0000 7ff90c027640 1 --2- 172.21.15.16:0/4112148363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90415ff00 0x7ff9040cc820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff904073300 tx=0x7ff8f402a270 comp rx=0 tx=0).stop 2024-09-12T01:33:31.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.388+0000 7ff90c027640 1 -- 172.21.15.16:0/4112148363 shutdown_connections 2024-09-12T01:33:31.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.388+0000 7ff90c027640 1 --2- 172.21.15.16:0/4112148363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90415ff00 0x7ff9040cc820 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.388+0000 7ff90c027640 1 --2- 172.21.15.16:0/4112148363 >> v2:172.21.15.33:3300/0 conn(0x7ff904156940 0x7ff90415d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.388+0000 7ff90c027640 1 -- 172.21.15.16:0/4112148363 >> 172.21.15.16:0/4112148363 conn(0x7ff90406fdc0 msgr2=0x7ff9040701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.388+0000 7ff90c027640 1 -- 172.21.15.16:0/4112148363 shutdown_connections 2024-09-12T01:33:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.388+0000 7ff90c027640 1 -- 172.21.15.16:0/4112148363 wait complete. 2024-09-12T01:33:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.389+0000 7ff90c027640 1 Processor -- start 2024-09-12T01:33:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.389+0000 7ff90c027640 1 -- start start 2024-09-12T01:33:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff90c027640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff904156940 0x7ff90406dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff90c027640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90415ff00 0x7ff90406e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff90c027640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff904160390 0x7ff90406e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff90c027640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9040d19b0 con 0x7ff90415ff00 2024-09-12T01:33:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff90c027640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff904156d20 con 0x7ff904160390 2024-09-12T01:33:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff90c027640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff9040d1b30 con 0x7ff904156940 2024-09-12T01:33:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff9095c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90415ff00 0x7ff90406e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff9095c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90415ff00 0x7ff90406e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41998/0 (socket says 172.21.15.16:41998) 2024-09-12T01:33:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff9095c2640 1 -- 172.21.15.16:0/3636872823 learned_addr learned my addr 172.21.15.16:0/3636872823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff909dc3640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff904156940 0x7ff90406dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff90a5c4640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff904160390 0x7ff90406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff9095c2640 1 -- 172.21.15.16:0/3636872823 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff904156940 msgr2=0x7ff90406dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff9095c2640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff904156940 0x7ff90406dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff9095c2640 1 -- 172.21.15.16:0/3636872823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff904160390 msgr2=0x7ff90406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff9095c2640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff904160390 0x7ff90406e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.390+0000 7ff9095c2640 1 -- 172.21.15.16:0/3636872823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90414ee20 con 0x7ff90415ff00 2024-09-12T01:33:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.391+0000 7ff9095c2640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90415ff00 0x7ff90406e1e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4000c00 tx=0x7ff8f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.391+0000 7ff8fa7fc640 1 -- 172.21.15.16:0/3636872823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4037040 con 0x7ff90415ff00 2024-09-12T01:33:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.391+0000 7ff8fa7fc640 1 -- 172.21.15.16:0/3636872823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f4042070 con 0x7ff90415ff00 2024-09-12T01:33:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.391+0000 7ff90c027640 1 -- 172.21.15.16:0/3636872823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff90406d520 con 0x7ff90415ff00 2024-09-12T01:33:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.391+0000 7ff8fa7fc640 1 -- 172.21.15.16:0/3636872823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4030880 con 0x7ff90415ff00 2024-09-12T01:33:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.391+0000 7ff90c027640 1 -- 172.21.15.16:0/3636872823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff90414db70 con 0x7ff90415ff00 2024-09-12T01:33:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.392+0000 7ff90c027640 1 -- 172.21.15.16:0/3636872823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff90414de00 con 0x7ff90415ff00 2024-09-12T01:33:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.393+0000 7ff8fa7fc640 1 -- 172.21.15.16:0/3636872823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8f4030a20 con 0x7ff90415ff00 2024-09-12T01:33:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.393+0000 7ff8fa7fc640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff8e806be30 0x7ff8e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.393+0000 7ff8fa7fc640 1 -- 172.21.15.16:0/3636872823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff8f40ab900 con 0x7ff90415ff00 2024-09-12T01:33:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.393+0000 7ff909dc3640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff8e806be30 0x7ff8e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.394+0000 7ff909dc3640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff8e806be30 0x7ff8e806e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff900009b40 tx=0x7ff900009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:31.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.398+0000 7ff8fa7fc640 1 -- 172.21.15.16:0/3636872823 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff8f4079920 con 0x7ff90415ff00 2024-09-12T01:33:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.608+0000 7ff90c027640 1 -- 172.21.15.16:0/3636872823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff90415d320 con 0x7ff90415ff00 2024-09-12T01:33:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.609+0000 7ff8fa7fc640 1 -- 172.21.15.16:0/3636872823 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7ff8f407dc20 con 0x7ff90415ff00 2024-09-12T01:33:31.609 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:31.609 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-12T01:33:31:251113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4157213575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4157213575},{"type":"v1","addr":"172.21.15.16:6835","nonce":4157213575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1554674078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1554674078},{"type":"v1","addr":"172.21.15.16:6837","nonce":1554674078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/359404074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":359404074},{"type":"v1","addr":"172.21.15.33:6834","nonce":359404074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:33:11.492995+0000","modified":"2024-09-12T01:33:30.603230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2179301157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2179301157},{"type":"v1","addr":"172.21.15.33:6835","nonce":2179301157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-12T01:33:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-12T01:33:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.611+0000 7ff90c027640 1 -- 172.21.15.16:0/3636872823 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff8e806be30 msgr2=0x7ff8e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.611+0000 7ff90c027640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff8e806be30 0x7ff8e806e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff900009b40 tx=0x7ff900009190 comp rx=0 tx=0).stop 2024-09-12T01:33:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.611+0000 7ff90c027640 1 -- 172.21.15.16:0/3636872823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90415ff00 msgr2=0x7ff90406e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.611+0000 7ff90c027640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90415ff00 0x7ff90406e1e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4000c00 tx=0x7ff8f40046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.611+0000 7ff90c027640 1 -- 172.21.15.16:0/3636872823 shutdown_connections 2024-09-12T01:33:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.611+0000 7ff90c027640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff8e806be30 0x7ff8e806e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.611+0000 7ff90c027640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff904160390 0x7ff90406e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.611+0000 7ff90c027640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90415ff00 0x7ff90406e1e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.611+0000 7ff90c027640 1 --2- 172.21.15.16:0/3636872823 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff904156940 0x7ff90406dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.611+0000 7ff90c027640 1 -- 172.21.15.16:0/3636872823 >> 172.21.15.16:0/3636872823 conn(0x7ff90406fdc0 msgr2=0x7ff9040712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.611+0000 7ff90c027640 1 -- 172.21.15.16:0/3636872823 shutdown_connections 2024-09-12T01:33:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.611+0000 7ff90c027640 1 -- 172.21.15.16:0/3636872823 wait complete. 2024-09-12T01:33:31.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:33:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.725+0000 7fb873245640 1 Processor -- start 2024-09-12T01:33:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.725+0000 7fb873245640 1 -- start start 2024-09-12T01:33:31.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb873245640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb86c15e760 0x7fb86c0cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:31.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb873245640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb86c0b47f0 0x7fb86c0b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:31.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb873245640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb86c073ec0 con 0x7fb86c06baf0 2024-09-12T01:33:31.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb873245640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb86c072e20 con 0x7fb86c0b47f0 2024-09-12T01:33:31.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb873245640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb86c073100 con 0x7fb86c15e760 2024-09-12T01:33:31.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb8717e2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb86c0b47f0 0x7fb86c0b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:31.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb8717e2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb86c0b47f0 0x7fb86c0b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42004/0 (socket says 172.21.15.16:42004) 2024-09-12T01:33:31.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb870fe1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb86c06baf0 0x7fb86c06bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53570/0 (socket says 172.21.15.16:53570) 2024-09-12T01:33:31.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb8717e2640 1 -- 172.21.15.16:0/3462328330 learned_addr learned my addr 172.21.15.16:0/3462328330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:31.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb86bfff640 1 --2- 172.21.15.16:0/3462328330 >> v2:172.21.15.33:3300/0 conn(0x7fb86c15e760 0x7fb86c0cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:31.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb8717e2640 1 -- 172.21.15.16:0/3462328330 >> v2:172.21.15.33:3300/0 conn(0x7fb86c15e760 msgr2=0x7fb86c0cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:31.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb8717e2640 1 --2- 172.21.15.16:0/3462328330 >> v2:172.21.15.33:3300/0 conn(0x7fb86c15e760 0x7fb86c0cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb8717e2640 1 -- 172.21.15.16:0/3462328330 >> v1:172.21.15.16:6789/0 conn(0x7fb86c06baf0 legacy=0x7fb86c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb8717e2640 1 -- 172.21.15.16:0/3462328330 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c0733e0 con 0x7fb86c0b47f0 2024-09-12T01:33:31.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb86b7fe640 1 -- 172.21.15.16:0/3462328330 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1212199866 0 0) 0x7fb86c073ec0 con 0x7fb86c06baf0 2024-09-12T01:33:31.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.726+0000 7fb86bfff640 1 --2- 172.21.15.16:0/3462328330 >> v2:172.21.15.33:3300/0 conn(0x7fb86c15e760 0x7fb86c0cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:31.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.727+0000 7fb8717e2640 1 --2- 172.21.15.16:0/3462328330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86c0b47f0 0x7fb86c0b4c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0d0d00 tx=0x7fb86402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42c0eef20646aae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:31.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.727+0000 7fb86b7fe640 1 -- 172.21.15.16:0/3462328330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864037070 con 0x7fb86c0b47f0 2024-09-12T01:33:31.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.727+0000 7fb86b7fe640 1 -- 172.21.15.16:0/3462328330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb864032440 con 0x7fb86c0b47f0 2024-09-12T01:33:31.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.727+0000 7fb86b7fe640 1 -- 172.21.15.16:0/3462328330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864032810 con 0x7fb86c0b47f0 2024-09-12T01:33:31.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.727+0000 7fb873245640 1 -- 172.21.15.16:0/3462328330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86c0b47f0 msgr2=0x7fb86c0b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.727+0000 7fb873245640 1 --2- 172.21.15.16:0/3462328330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86c0b47f0 0x7fb86c0b4c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0d0d00 tx=0x7fb86402a1e0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.727+0000 7fb873245640 1 -- 172.21.15.16:0/3462328330 shutdown_connections 2024-09-12T01:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.727+0000 7fb873245640 1 --2- 172.21.15.16:0/3462328330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86c0b47f0 0x7fb86c0b4c80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.727+0000 7fb873245640 1 --2- 172.21.15.16:0/3462328330 >> v2:172.21.15.33:3300/0 conn(0x7fb86c15e760 0x7fb86c0cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.727+0000 7fb873245640 1 -- 172.21.15.16:0/3462328330 >> 172.21.15.16:0/3462328330 conn(0x7fb86c0b2c50 msgr2=0x7fb86c0b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.728+0000 7fb873245640 1 -- 172.21.15.16:0/3462328330 shutdown_connections 2024-09-12T01:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.728+0000 7fb873245640 1 -- 172.21.15.16:0/3462328330 wait complete. 2024-09-12T01:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.729+0000 7fb873245640 1 Processor -- start 2024-09-12T01:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.729+0000 7fb873245640 1 -- start start 2024-09-12T01:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb873245640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c06baf0 0x7fb86c146970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb873245640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb86c0b47f0 0x7fb86c1608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb873245640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86c15e760 0x7fb86c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb873245640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c06c850 con 0x7fb86c15e760 2024-09-12T01:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb873245640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c0739a0 con 0x7fb86c06baf0 2024-09-12T01:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb873245640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb86c0cf520 con 0x7fb86c0b47f0 2024-09-12T01:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb8717e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86c15e760 0x7fb86c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb8717e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86c15e760 0x7fb86c15ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42020/0 (socket says 172.21.15.16:42020) 2024-09-12T01:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb8717e2640 1 -- 172.21.15.16:0/2293137515 learned_addr learned my addr 172.21.15.16:0/2293137515 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb870fe1640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c06baf0 0x7fb86c146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb86bfff640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb86c0b47f0 0x7fb86c1608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb8717e2640 1 -- 172.21.15.16:0/2293137515 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb86c0b47f0 msgr2=0x7fb86c1608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb8717e2640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb86c0b47f0 0x7fb86c1608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb8717e2640 1 -- 172.21.15.16:0/2293137515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c06baf0 msgr2=0x7fb86c146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb8717e2640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c06baf0 0x7fb86c146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb8717e2640 1 -- 172.21.15.16:0/2293137515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c1471f0 con 0x7fb86c15e760 2024-09-12T01:33:31.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb8717e2640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86c15e760 0x7fb86c15ec90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb86402a220 tx=0x7fb8640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:31.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb8697fa640 1 -- 172.21.15.16:0/2293137515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864037040 con 0x7fb86c15e760 2024-09-12T01:33:31.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb8697fa640 1 -- 172.21.15.16:0/2293137515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb864004250 con 0x7fb86c15e760 2024-09-12T01:33:31.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb873245640 1 -- 172.21.15.16:0/2293137515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c1474d0 con 0x7fb86c15e760 2024-09-12T01:33:31.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.730+0000 7fb8697fa640 1 -- 172.21.15.16:0/2293137515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86403c670 con 0x7fb86c15e760 2024-09-12T01:33:31.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.731+0000 7fb873245640 1 -- 172.21.15.16:0/2293137515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c147a90 con 0x7fb86c15e760 2024-09-12T01:33:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.732+0000 7fb873245640 1 -- 172.21.15.16:0/2293137515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86c0cefe0 con 0x7fb86c15e760 2024-09-12T01:33:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.732+0000 7fb8697fa640 1 -- 172.21.15.16:0/2293137515 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb86403a070 con 0x7fb86c15e760 2024-09-12T01:33:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.732+0000 7fb8697fa640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb84c06be30 0x7fb84c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.732+0000 7fb8697fa640 1 -- 172.21.15.16:0/2293137515 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb8640ab9d0 con 0x7fb86c15e760 2024-09-12T01:33:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.732+0000 7fb870fe1640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb84c06be30 0x7fb84c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.733+0000 7fb870fe1640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb84c06be30 0x7fb84c06e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb854007fd0 tx=0x7fb854007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:31.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.737+0000 7fb8697fa640 1 -- 172.21.15.16:0/2293137515 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb864041020 con 0x7fb86c15e760 2024-09-12T01:33:31.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.912+0000 7fb873245640 1 -- 172.21.15.16:0/2293137515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb86c0b38b0 con 0x7fb86c15e760 2024-09-12T01:33:31.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.913+0000 7fb8697fa640 1 -- 172.21.15.16:0/2293137515 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18574 (secure 0 0 0) 0x7fb864079a80 con 0x7fb86c15e760 2024-09-12T01:33:31.914 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:31.914 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:33:18.213622+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:33:09.364639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:33:10.478237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:33:14.204624+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:33:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.916+0000 7fb873245640 1 -- 172.21.15.16:0/2293137515 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb84c06be30 msgr2=0x7fb84c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.916+0000 7fb873245640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb84c06be30 0x7fb84c06e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb854007fd0 tx=0x7fb854007de0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.916+0000 7fb873245640 1 -- 172.21.15.16:0/2293137515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86c15e760 msgr2=0x7fb86c15ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.916+0000 7fb873245640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86c15e760 0x7fb86c15ec90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb86402a220 tx=0x7fb8640046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.916+0000 7fb873245640 1 -- 172.21.15.16:0/2293137515 shutdown_connections 2024-09-12T01:33:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.916+0000 7fb873245640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb84c06be30 0x7fb84c06e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.916+0000 7fb873245640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86c15e760 0x7fb86c15ec90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.916+0000 7fb873245640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb86c0b47f0 0x7fb86c1608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.916+0000 7fb873245640 1 --2- 172.21.15.16:0/2293137515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c06baf0 0x7fb86c146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.916+0000 7fb873245640 1 -- 172.21.15.16:0/2293137515 >> 172.21.15.16:0/2293137515 conn(0x7fb86c0b2c50 msgr2=0x7fb86c0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.916+0000 7fb873245640 1 -- 172.21.15.16:0/2293137515 shutdown_connections 2024-09-12T01:33:31.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:31.916+0000 7fb873245640 1 -- 172.21.15.16:0/2293137515 wait complete. 2024-09-12T01:33:31.927 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T01:33:31.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:33:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.029+0000 7fcae9e82640 1 Processor -- start 2024-09-12T01:33:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.029+0000 7fcae9e82640 1 -- start start 2024-09-12T01:33:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.030+0000 7fcae9e82640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcae415e510 0x7fcae40ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.030+0000 7fcae9e82640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcae40d13d0 0x7fcae40b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.030+0000 7fcae9e82640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae40d3430 con 0x7fcae40cf040 2024-09-12T01:33:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.030+0000 7fcae9e82640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fcae40d36d0 con 0x7fcae415e510 2024-09-12T01:33:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.030+0000 7fcae9e82640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fcae40d39b0 con 0x7fcae40d13d0 2024-09-12T01:33:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.030+0000 7fcae2ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcae40cf040 0x7fcae40d0cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53580/0 (socket says 172.21.15.16:53580) 2024-09-12T01:33:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.030+0000 7fcae2ffd640 1 -- 172.21.15.16:0/4012572564 learned_addr learned my addr 172.21.15.16:0/4012572564 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.030+0000 7fcae3fff640 1 --2- 172.21.15.16:0/4012572564 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcae40d13d0 0x7fcae40b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae37fe640 1 --2- 172.21.15.16:0/4012572564 >> v2:172.21.15.33:3300/0 conn(0x7fcae415e510 0x7fcae40ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 40728346 0 0) 0x7fcae40d3430 con 0x7fcae40cf040 2024-09-12T01:33:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcacc003520 con 0x7fcae40cf040 2024-09-12T01:33:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2932002099 0 0) 0x7fcacc003520 con 0x7fcae40cf040 2024-09-12T01:33:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcae40d13d0 msgr2=0x7fcae40b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 --2- 172.21.15.16:0/4012572564 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcae40d13d0 0x7fcae40b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 >> v2:172.21.15.33:3300/0 conn(0x7fcae415e510 msgr2=0x7fcae40ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 --2- 172.21.15.16:0/4012572564 >> v2:172.21.15.33:3300/0 conn(0x7fcae415e510 0x7fcae40ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae40d3c90 con 0x7fcae40cf040 2024-09-12T01:33:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7fcad8001670 con 0x7fcae40cf040 2024-09-12T01:33:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 >> v1:172.21.15.16:6789/0 conn(0x7fcae40cf040 legacy=0x7fcae40d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 --2- 172.21.15.16:0/4012572564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcacc003a20 0x7fcacc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 --2- 172.21.15.16:0/4012572564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc004800 0x7fcacc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 --2- 172.21.15.16:0/4012572564 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcae40d13d0 0x7fcacc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcae40d36d0 con 0x7fcacc004800 2024-09-12T01:33:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcae40d39b0 con 0x7fcacc003a20 2024-09-12T01:33:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fcacc00ba70 con 0x7fcae40d13d0 2024-09-12T01:33:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae2ffd640 1 --2- 172.21.15.16:0/4012572564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc004800 0x7fcacc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae37fe640 1 --2- 172.21.15.16:0/4012572564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcacc003a20 0x7fcacc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae3fff640 1 --2- 172.21.15.16:0/4012572564 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcae40d13d0 0x7fcacc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae2ffd640 1 -- 172.21.15.16:0/4012572564 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcae40d13d0 msgr2=0x7fcacc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae2ffd640 1 --2- 172.21.15.16:0/4012572564 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcae40d13d0 0x7fcacc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae2ffd640 1 -- 172.21.15.16:0/4012572564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcacc003a20 msgr2=0x7fcacc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae2ffd640 1 --2- 172.21.15.16:0/4012572564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcacc003a20 0x7fcacc003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.031+0000 7fcae2ffd640 1 -- 172.21.15.16:0/4012572564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcacc003520 con 0x7fcacc004800 2024-09-12T01:33:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.032+0000 7fcae2ffd640 1 --2- 172.21.15.16:0/4012572564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc004800 0x7fcacc004c80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcad8001050 tx=0x7fcacc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f19e2219c5e6b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.032+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad802e070 con 0x7fcacc004800 2024-09-12T01:33:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.032+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad8005af0 con 0x7fcacc004800 2024-09-12T01:33:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.032+0000 7fcae27fc640 1 -- 172.21.15.16:0/4012572564 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8007d70 con 0x7fcacc004800 2024-09-12T01:33:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.033+0000 7fcae9e82640 1 -- 172.21.15.16:0/4012572564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc004800 msgr2=0x7fcacc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.033+0000 7fcae9e82640 1 --2- 172.21.15.16:0/4012572564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc004800 0x7fcacc004c80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcad8001050 tx=0x7fcacc005830 comp rx=0 tx=0).stop 2024-09-12T01:33:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.033+0000 7fcae3fff640 1 -- 172.21.15.16:0/4012572564 reap_dead start 2024-09-12T01:33:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.033+0000 7fcae9e82640 1 -- 172.21.15.16:0/4012572564 shutdown_connections 2024-09-12T01:33:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.033+0000 7fcae9e82640 1 -- 172.21.15.16:0/4012572564 >> 172.21.15.16:0/4012572564 conn(0x7fcae406d1a0 msgr2=0x7fcae4073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.033+0000 7fcae9e82640 1 -- 172.21.15.16:0/4012572564 shutdown_connections 2024-09-12T01:33:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.033+0000 7fcae9e82640 1 -- 172.21.15.16:0/4012572564 wait complete. 2024-09-12T01:33:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.034+0000 7fcae9e82640 1 Processor -- start 2024-09-12T01:33:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.034+0000 7fcae9e82640 1 -- start start 2024-09-12T01:33:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae9e82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc004800 0x7fcae41f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae9e82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae41f4310 0x7fcae40aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae9e82640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcae40ab380 0x7fcae40cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae9e82640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcae415e8f0 con 0x7fcacc004800 2024-09-12T01:33:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae9e82640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcae40d22a0 con 0x7fcae41f4310 2024-09-12T01:33:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae9e82640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fcae415ea70 con 0x7fcae40ab380 2024-09-12T01:33:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc004800 0x7fcae41f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc004800 0x7fcae41f7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42042/0 (socket says 172.21.15.16:42042) 2024-09-12T01:33:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae37fe640 1 -- 172.21.15.16:0/2381073856 learned_addr learned my addr 172.21.15.16:0/2381073856 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae37fe640 1 -- 172.21.15.16:0/2381073856 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcae40ab380 msgr2=0x7fcae40cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae37fe640 1 --2- 172.21.15.16:0/2381073856 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcae40ab380 0x7fcae40cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae37fe640 1 -- 172.21.15.16:0/2381073856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae41f4310 msgr2=0x7fcae40aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae37fe640 1 --2- 172.21.15.16:0/2381073856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae41f4310 0x7fcae40aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.035+0000 7fcae37fe640 1 -- 172.21.15.16:0/2381073856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae41f4770 con 0x7fcacc004800 2024-09-12T01:33:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.036+0000 7fcae37fe640 1 --2- 172.21.15.16:0/2381073856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc004800 0x7fcae41f7210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcad4011960 tx=0x7fcad4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.036+0000 7fcacbfff640 1 -- 172.21.15.16:0/2381073856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad400fa10 con 0x7fcacc004800 2024-09-12T01:33:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.036+0000 7fcacbfff640 1 -- 172.21.15.16:0/2381073856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad400fbb0 con 0x7fcacc004800 2024-09-12T01:33:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.036+0000 7fcae9e82640 1 -- 172.21.15.16:0/2381073856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae40cc880 con 0x7fcacc004800 2024-09-12T01:33:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.036+0000 7fcacbfff640 1 -- 172.21.15.16:0/2381073856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4014440 con 0x7fcacc004800 2024-09-12T01:33:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.036+0000 7fcae9e82640 1 -- 172.21.15.16:0/2381073856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae40cce40 con 0x7fcacc004800 2024-09-12T01:33:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.037+0000 7fcae9e82640 1 -- 172.21.15.16:0/2381073856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae40ceb00 con 0x7fcacc004800 2024-09-12T01:33:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.037+0000 7fcacbfff640 1 -- 172.21.15.16:0/2381073856 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcad4012040 con 0x7fcacc004800 2024-09-12T01:33:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.038+0000 7fcacbfff640 1 --2- 172.21.15.16:0/2381073856 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fcad006be30 0x7fcad006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.038+0000 7fcacbfff640 1 -- 172.21.15.16:0/2381073856 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fcad408b6c0 con 0x7fcacc004800 2024-09-12T01:33:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.038+0000 7fcae2ffd640 1 --2- 172.21.15.16:0/2381073856 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fcad006be30 0x7fcad006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.038+0000 7fcae2ffd640 1 --2- 172.21.15.16:0/2381073856 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fcad006be30 0x7fcad006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcad8033000 tx=0x7fcad8007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.042+0000 7fcacbfff640 1 -- 172.21.15.16:0/2381073856 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcad4019030 con 0x7fcacc004800 2024-09-12T01:33:32.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.216+0000 7fcae9e82640 1 -- 172.21.15.16:0/2381073856 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcae40cfaa0 con 0x7fcad006be30 2024-09-12T01:33:32.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.225+0000 7fcacbfff640 1 -- 172.21.15.16:0/2381073856 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fcae40cfaa0 con 0x7fcad006be30 2024-09-12T01:33:32.229 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:32.231 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:33:32.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.229+0000 7fcae9e82640 1 -- 172.21.15.16:0/2381073856 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fcad006be30 msgr2=0x7fcad006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.229+0000 7fcae9e82640 1 --2- 172.21.15.16:0/2381073856 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fcad006be30 0x7fcad006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcad8033000 tx=0x7fcad8007000 comp rx=0 tx=0).stop 2024-09-12T01:33:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.229+0000 7fcae9e82640 1 -- 172.21.15.16:0/2381073856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc004800 msgr2=0x7fcae41f7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.229+0000 7fcae9e82640 1 --2- 172.21.15.16:0/2381073856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc004800 0x7fcae41f7210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcad4011960 tx=0x7fcad4011e20 comp rx=0 tx=0).stop 2024-09-12T01:33:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.230+0000 7fcae9e82640 1 -- 172.21.15.16:0/2381073856 shutdown_connections 2024-09-12T01:33:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.230+0000 7fcae9e82640 1 --2- 172.21.15.16:0/2381073856 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fcad006be30 0x7fcad006e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.230+0000 7fcae9e82640 1 --2- 172.21.15.16:0/2381073856 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcae40ab380 0x7fcae40cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.230+0000 7fcae9e82640 1 --2- 172.21.15.16:0/2381073856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcae41f4310 0x7fcae40aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.230+0000 7fcae9e82640 1 --2- 172.21.15.16:0/2381073856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc004800 0x7fcae41f7210 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.230+0000 7fcae9e82640 1 -- 172.21.15.16:0/2381073856 >> 172.21.15.16:0/2381073856 conn(0x7fcae406d1a0 msgr2=0x7fcae4074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.230+0000 7fcae9e82640 1 -- 172.21.15.16:0/2381073856 shutdown_connections 2024-09-12T01:33:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.230+0000 7fcae9e82640 1 -- 172.21.15.16:0/2381073856 wait complete. 2024-09-12T01:33:32.240 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-12T01:33:30.474383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236204,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738516,"statfs":{"total":773094113280,"available":772852240384,"internally_reserved":0,"allocated":3416064,"data_stored":2069540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002863"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413650+0000","last_change":"2024-09-12T01:33:11.148986+0000","last_active":"2024-09-12T01:33:19.413650+0000","last_peered":"2024-09-12T01:33:19.413650+0000","last_clean":"2024-09-12T01:33:19.413650+0000","last_became_active":"2024-09-12T01:33:11.146323+0000","last_became_peered":"2024-09-12T01:33:11.146323+0000","last_unstale":"2024-09-12T01:33:19.413650+0000","last_undegraded":"2024-09-12T01:33:19.413650+0000","last_fullsized":"2024-09-12T01:33:19.413650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:54.118009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217034+0000","last_change":"2024-09-12T01:33:12.183084+0000","last_active":"2024-09-12T01:33:18.217034+0000","last_peered":"2024-09-12T01:33:18.217034+0000","last_clean":"2024-09-12T01:33:18.217034+0000","last_became_active":"2024-09-12T01:33:12.182286+0000","last_became_peered":"2024-09-12T01:33:12.182286+0000","last_unstale":"2024-09-12T01:33:18.217034+0000","last_undegraded":"2024-09-12T01:33:18.217034+0000","last_fullsized":"2024-09-12T01:33:18.217034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:46.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216567+0000","last_change":"2024-09-12T01:33:16.593060+0000","last_active":"2024-09-12T01:33:18.216567+0000","last_peered":"2024-09-12T01:33:18.216567+0000","last_clean":"2024-09-12T01:33:18.216567+0000","last_became_active":"2024-09-12T01:33:16.592584+0000","last_became_peered":"2024-09-12T01:33:16.592584+0000","last_unstale":"2024-09-12T01:33:18.216567+0000","last_undegraded":"2024-09-12T01:33:18.216567+0000","last_fullsized":"2024-09-12T01:33:18.216567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:57.525309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217133+0000","last_change":"2024-09-12T01:33:11.142022+0000","last_active":"2024-09-12T01:33:18.217133+0000","last_peered":"2024-09-12T01:33:18.217133+0000","last_clean":"2024-09-12T01:33:18.217133+0000","last_became_active":"2024-09-12T01:33:11.141709+0000","last_became_peered":"2024-09-12T01:33:11.141709+0000","last_unstale":"2024-09-12T01:33:18.217133+0000","last_undegraded":"2024-09-12T01:33:18.217133+0000","last_fullsized":"2024-09-12T01:33:18.217133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:04.872785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413419+0000","last_change":"2024-09-12T01:33:12.176163+0000","last_active":"2024-09-12T01:33:19.413419+0000","last_peered":"2024-09-12T01:33:19.413419+0000","last_clean":"2024-09-12T01:33:19.413419+0000","last_became_active":"2024-09-12T01:33:12.175795+0000","last_became_peered":"2024-09-12T01:33:12.175795+0000","last_unstale":"2024-09-12T01:33:19.413419+0000","last_undegraded":"2024-09-12T01:33:19.413419+0000","last_fullsized":"2024-09-12T01:33:19.413419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:29.557033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217872+0000","last_change":"2024-09-12T01:33:16.202792+0000","last_active":"2024-09-12T01:33:18.217872+0000","last_peered":"2024-09-12T01:33:18.217872+0000","last_clean":"2024-09-12T01:33:18.217872+0000","last_became_active":"2024-09-12T01:33:16.202352+0000","last_became_peered":"2024-09-12T01:33:16.202352+0000","last_unstale":"2024-09-12T01:33:18.217872+0000","last_undegraded":"2024-09-12T01:33:18.217872+0000","last_fullsized":"2024-09-12T01:33:18.217872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:39.155565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487520+0000","last_change":"2024-09-12T01:33:11.149265+0000","last_active":"2024-09-12T01:33:19.487520+0000","last_peered":"2024-09-12T01:33:19.487520+0000","last_clean":"2024-09-12T01:33:19.487520+0000","last_became_active":"2024-09-12T01:33:11.148975+0000","last_became_peered":"2024-09-12T01:33:11.148975+0000","last_unstale":"2024-09-12T01:33:19.487520+0000","last_undegraded":"2024-09-12T01:33:19.487520+0000","last_fullsized":"2024-09-12T01:33:19.487520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:49.680053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839630+0000","last_change":"2024-09-12T01:33:12.175597+0000","last_active":"2024-09-12T01:33:18.839630+0000","last_peered":"2024-09-12T01:33:18.839630+0000","last_clean":"2024-09-12T01:33:18.839630+0000","last_became_active":"2024-09-12T01:33:12.175271+0000","last_became_peered":"2024-09-12T01:33:12.175271+0000","last_unstale":"2024-09-12T01:33:18.839630+0000","last_undegraded":"2024-09-12T01:33:18.839630+0000","last_fullsized":"2024-09-12T01:33:18.839630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:24.363602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218697+0000","last_change":"2024-09-12T01:33:16.200812+0000","last_active":"2024-09-12T01:33:18.218697+0000","last_peered":"2024-09-12T01:33:18.218697+0000","last_clean":"2024-09-12T01:33:18.218697+0000","last_became_active":"2024-09-12T01:33:16.200029+0000","last_became_peered":"2024-09-12T01:33:16.200029+0000","last_unstale":"2024-09-12T01:33:18.218697+0000","last_undegraded":"2024-09-12T01:33:18.218697+0000","last_fullsized":"2024-09-12T01:33:18.218697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:19.443781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485752+0000","last_change":"2024-09-12T01:33:11.153223+0000","last_active":"2024-09-12T01:33:19.485752+0000","last_peered":"2024-09-12T01:33:19.485752+0000","last_clean":"2024-09-12T01:33:19.485752+0000","last_became_active":"2024-09-12T01:33:11.152938+0000","last_became_peered":"2024-09-12T01:33:11.152938+0000","last_unstale":"2024-09-12T01:33:19.485752+0000","last_undegraded":"2024-09-12T01:33:19.485752+0000","last_fullsized":"2024-09-12T01:33:19.485752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839743+0000","last_change":"2024-09-12T01:33:12.178087+0000","last_active":"2024-09-12T01:33:18.839743+0000","last_peered":"2024-09-12T01:33:18.839743+0000","last_clean":"2024-09-12T01:33:18.839743+0000","last_became_active":"2024-09-12T01:33:12.176556+0000","last_became_peered":"2024-09-12T01:33:12.176556+0000","last_unstale":"2024-09-12T01:33:18.839743+0000","last_undegraded":"2024-09-12T01:33:18.839743+0000","last_fullsized":"2024-09-12T01:33:18.839743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:48.292206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218149+0000","last_change":"2024-09-12T01:33:16.193585+0000","last_active":"2024-09-12T01:33:18.218149+0000","last_peered":"2024-09-12T01:33:18.218149+0000","last_clean":"2024-09-12T01:33:18.218149+0000","last_became_active":"2024-09-12T01:33:16.193050+0000","last_became_peered":"2024-09-12T01:33:16.193050+0000","last_unstale":"2024-09-12T01:33:18.218149+0000","last_undegraded":"2024-09-12T01:33:18.218149+0000","last_fullsized":"2024-09-12T01:33:18.218149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:40.563121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486654+0000","last_change":"2024-09-12T01:33:12.178078+0000","last_active":"2024-09-12T01:33:19.486654+0000","last_peered":"2024-09-12T01:33:19.486654+0000","last_clean":"2024-09-12T01:33:19.486654+0000","last_became_active":"2024-09-12T01:33:12.177806+0000","last_became_peered":"2024-09-12T01:33:12.177806+0000","last_unstale":"2024-09-12T01:33:19.486654+0000","last_undegraded":"2024-09-12T01:33:19.486654+0000","last_fullsized":"2024-09-12T01:33:19.486654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:25.007983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485717+0000","last_change":"2024-09-12T01:33:11.149750+0000","last_active":"2024-09-12T01:33:19.485717+0000","last_peered":"2024-09-12T01:33:19.485717+0000","last_clean":"2024-09-12T01:33:19.485717+0000","last_became_active":"2024-09-12T01:33:11.149442+0000","last_became_peered":"2024-09-12T01:33:11.149442+0000","last_unstale":"2024-09-12T01:33:19.485717+0000","last_undegraded":"2024-09-12T01:33:19.485717+0000","last_fullsized":"2024-09-12T01:33:19.485717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218830+0000","last_change":"2024-09-12T01:33:16.202498+0000","last_active":"2024-09-12T01:33:18.218830+0000","last_peered":"2024-09-12T01:33:18.218830+0000","last_clean":"2024-09-12T01:33:18.218830+0000","last_became_active":"2024-09-12T01:33:16.200800+0000","last_became_peered":"2024-09-12T01:33:16.200800+0000","last_unstale":"2024-09-12T01:33:18.218830+0000","last_undegraded":"2024-09-12T01:33:18.218830+0000","last_fullsized":"2024-09-12T01:33:18.218830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:53.648825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216927+0000","last_change":"2024-09-12T01:33:12.169842+0000","last_active":"2024-09-12T01:33:18.216927+0000","last_peered":"2024-09-12T01:33:18.216927+0000","last_clean":"2024-09-12T01:33:18.216927+0000","last_became_active":"2024-09-12T01:33:12.169524+0000","last_became_peered":"2024-09-12T01:33:12.169524+0000","last_unstale":"2024-09-12T01:33:18.216927+0000","last_undegraded":"2024-09-12T01:33:18.216927+0000","last_fullsized":"2024-09-12T01:33:18.216927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:38.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485299+0000","last_change":"2024-09-12T01:33:11.147221+0000","last_active":"2024-09-12T01:33:19.485299+0000","last_peered":"2024-09-12T01:33:19.485299+0000","last_clean":"2024-09-12T01:33:19.485299+0000","last_became_active":"2024-09-12T01:33:11.146928+0000","last_became_peered":"2024-09-12T01:33:11.146928+0000","last_unstale":"2024-09-12T01:33:19.485299+0000","last_undegraded":"2024-09-12T01:33:19.485299+0000","last_fullsized":"2024-09-12T01:33:19.485299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:22.885518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218288+0000","last_change":"2024-09-12T01:33:16.196589+0000","last_active":"2024-09-12T01:33:18.218288+0000","last_peered":"2024-09-12T01:33:18.218288+0000","last_clean":"2024-09-12T01:33:18.218288+0000","last_became_active":"2024-09-12T01:33:16.196092+0000","last_became_peered":"2024-09-12T01:33:16.196092+0000","last_unstale":"2024-09-12T01:33:18.218288+0000","last_undegraded":"2024-09-12T01:33:18.218288+0000","last_fullsized":"2024-09-12T01:33:18.218288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:28.321407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216715+0000","last_change":"2024-09-12T01:33:12.165178+0000","last_active":"2024-09-12T01:33:18.216715+0000","last_peered":"2024-09-12T01:33:18.216715+0000","last_clean":"2024-09-12T01:33:18.216715+0000","last_became_active":"2024-09-12T01:33:12.162823+0000","last_became_peered":"2024-09-12T01:33:12.162823+0000","last_unstale":"2024-09-12T01:33:18.216715+0000","last_undegraded":"2024-09-12T01:33:18.216715+0000","last_fullsized":"2024-09-12T01:33:18.216715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:54.679349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485555+0000","last_change":"2024-09-12T01:33:11.145358+0000","last_active":"2024-09-12T01:33:19.485555+0000","last_peered":"2024-09-12T01:33:19.485555+0000","last_clean":"2024-09-12T01:33:19.485555+0000","last_became_active":"2024-09-12T01:33:11.145035+0000","last_became_peered":"2024-09-12T01:33:11.145035+0000","last_unstale":"2024-09-12T01:33:19.485555+0000","last_undegraded":"2024-09-12T01:33:19.485555+0000","last_fullsized":"2024-09-12T01:33:19.485555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486406+0000","last_change":"2024-09-12T01:33:16.594344+0000","last_active":"2024-09-12T01:33:19.486406+0000","last_peered":"2024-09-12T01:33:19.486406+0000","last_clean":"2024-09-12T01:33:19.486406+0000","last_became_active":"2024-09-12T01:33:16.593980+0000","last_became_peered":"2024-09-12T01:33:16.593980+0000","last_unstale":"2024-09-12T01:33:19.486406+0000","last_undegraded":"2024-09-12T01:33:19.486406+0000","last_fullsized":"2024-09-12T01:33:19.486406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:14.088579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218224+0000","last_change":"2024-09-12T01:33:12.170635+0000","last_active":"2024-09-12T01:33:18.218224+0000","last_peered":"2024-09-12T01:33:18.218224+0000","last_clean":"2024-09-12T01:33:18.218224+0000","last_became_active":"2024-09-12T01:33:12.169588+0000","last_became_peered":"2024-09-12T01:33:12.169588+0000","last_unstale":"2024-09-12T01:33:18.218224+0000","last_undegraded":"2024-09-12T01:33:18.218224+0000","last_fullsized":"2024-09-12T01:33:18.218224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:50.002525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486441+0000","last_change":"2024-09-12T01:33:11.151147+0000","last_active":"2024-09-12T01:33:19.486441+0000","last_peered":"2024-09-12T01:33:19.486441+0000","last_clean":"2024-09-12T01:33:19.486441+0000","last_became_active":"2024-09-12T01:33:11.150915+0000","last_became_peered":"2024-09-12T01:33:11.150915+0000","last_unstale":"2024-09-12T01:33:19.486441+0000","last_undegraded":"2024-09-12T01:33:19.486441+0000","last_fullsized":"2024-09-12T01:33:19.486441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:56.100575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218568+0000","last_change":"2024-09-12T01:33:16.202342+0000","last_active":"2024-09-12T01:33:18.218568+0000","last_peered":"2024-09-12T01:33:18.218568+0000","last_clean":"2024-09-12T01:33:18.218568+0000","last_became_active":"2024-09-12T01:33:16.200265+0000","last_became_peered":"2024-09-12T01:33:16.200265+0000","last_unstale":"2024-09-12T01:33:18.218568+0000","last_undegraded":"2024-09-12T01:33:18.218568+0000","last_fullsized":"2024-09-12T01:33:18.218568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:13.593524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413572+0000","last_change":"2024-09-12T01:33:12.171306+0000","last_active":"2024-09-12T01:33:19.413572+0000","last_peered":"2024-09-12T01:33:19.413572+0000","last_clean":"2024-09-12T01:33:19.413572+0000","last_became_active":"2024-09-12T01:33:12.171026+0000","last_became_peered":"2024-09-12T01:33:12.171026+0000","last_unstale":"2024-09-12T01:33:19.413572+0000","last_undegraded":"2024-09-12T01:33:19.413572+0000","last_fullsized":"2024-09-12T01:33:19.413572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:02.272439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487629+0000","last_change":"2024-09-12T01:33:11.153620+0000","last_active":"2024-09-12T01:33:19.487629+0000","last_peered":"2024-09-12T01:33:19.487629+0000","last_clean":"2024-09-12T01:33:19.487629+0000","last_became_active":"2024-09-12T01:33:11.153278+0000","last_became_peered":"2024-09-12T01:33:11.153278+0000","last_unstale":"2024-09-12T01:33:19.487629+0000","last_undegraded":"2024-09-12T01:33:19.487629+0000","last_fullsized":"2024-09-12T01:33:19.487629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217980+0000","last_change":"2024-09-12T01:33:16.590320+0000","last_active":"2024-09-12T01:33:18.217980+0000","last_peered":"2024-09-12T01:33:18.217980+0000","last_clean":"2024-09-12T01:33:18.217980+0000","last_became_active":"2024-09-12T01:33:16.589806+0000","last_became_peered":"2024-09-12T01:33:16.589806+0000","last_unstale":"2024-09-12T01:33:18.217980+0000","last_undegraded":"2024-09-12T01:33:18.217980+0000","last_fullsized":"2024-09-12T01:33:18.217980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:01.750852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217345+0000","last_change":"2024-09-12T01:33:12.177022+0000","last_active":"2024-09-12T01:33:18.217345+0000","last_peered":"2024-09-12T01:33:18.217345+0000","last_clean":"2024-09-12T01:33:18.217345+0000","last_became_active":"2024-09-12T01:33:12.176544+0000","last_became_peered":"2024-09-12T01:33:12.176544+0000","last_unstale":"2024-09-12T01:33:18.217345+0000","last_undegraded":"2024-09-12T01:33:18.217345+0000","last_fullsized":"2024-09-12T01:33:18.217345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:51.254891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486161+0000","last_change":"2024-09-12T01:33:11.144439+0000","last_active":"2024-09-12T01:33:19.486161+0000","last_peered":"2024-09-12T01:33:19.486161+0000","last_clean":"2024-09-12T01:33:19.486161+0000","last_became_active":"2024-09-12T01:33:11.144134+0000","last_became_peered":"2024-09-12T01:33:11.144134+0000","last_unstale":"2024-09-12T01:33:19.486161+0000","last_undegraded":"2024-09-12T01:33:19.486161+0000","last_fullsized":"2024-09-12T01:33:19.486161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216491+0000","last_change":"2024-09-12T01:33:16.588984+0000","last_active":"2024-09-12T01:33:18.216491+0000","last_peered":"2024-09-12T01:33:18.216491+0000","last_clean":"2024-09-12T01:33:18.216491+0000","last_became_active":"2024-09-12T01:33:16.588531+0000","last_became_peered":"2024-09-12T01:33:16.588531+0000","last_unstale":"2024-09-12T01:33:18.216491+0000","last_undegraded":"2024-09-12T01:33:18.216491+0000","last_fullsized":"2024-09-12T01:33:18.216491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:35.903448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486153+0000","last_change":"2024-09-12T01:33:12.162118+0000","last_active":"2024-09-12T01:33:19.486153+0000","last_peered":"2024-09-12T01:33:19.486153+0000","last_clean":"2024-09-12T01:33:19.486153+0000","last_became_active":"2024-09-12T01:33:12.161817+0000","last_became_peered":"2024-09-12T01:33:12.161817+0000","last_unstale":"2024-09-12T01:33:19.486153+0000","last_undegraded":"2024-09-12T01:33:19.486153+0000","last_fullsized":"2024-09-12T01:33:19.486153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:54.679349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487612+0000","last_change":"2024-09-12T01:33:11.150724+0000","last_active":"2024-09-12T01:33:19.487612+0000","last_peered":"2024-09-12T01:33:19.487612+0000","last_clean":"2024-09-12T01:33:19.487612+0000","last_became_active":"2024-09-12T01:33:11.150444+0000","last_became_peered":"2024-09-12T01:33:11.150444+0000","last_unstale":"2024-09-12T01:33:19.487612+0000","last_undegraded":"2024-09-12T01:33:19.487612+0000","last_fullsized":"2024-09-12T01:33:19.487612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:26.579494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413397+0000","last_change":"2024-09-12T01:33:16.589570+0000","last_active":"2024-09-12T01:33:19.413397+0000","last_peered":"2024-09-12T01:33:19.413397+0000","last_clean":"2024-09-12T01:33:19.413397+0000","last_became_active":"2024-09-12T01:33:16.589137+0000","last_became_peered":"2024-09-12T01:33:16.589137+0000","last_unstale":"2024-09-12T01:33:19.413397+0000","last_undegraded":"2024-09-12T01:33:19.413397+0000","last_fullsized":"2024-09-12T01:33:19.413397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:31:18.804222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839884+0000","last_change":"2024-09-12T01:33:12.174625+0000","last_active":"2024-09-12T01:33:18.839884+0000","last_peered":"2024-09-12T01:33:18.839884+0000","last_clean":"2024-09-12T01:33:18.839884+0000","last_became_active":"2024-09-12T01:33:12.174308+0000","last_became_peered":"2024-09-12T01:33:12.174308+0000","last_unstale":"2024-09-12T01:33:18.839884+0000","last_undegraded":"2024-09-12T01:33:18.839884+0000","last_fullsized":"2024-09-12T01:33:18.839884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:24.134321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486236+0000","last_change":"2024-09-12T01:33:11.142280+0000","last_active":"2024-09-12T01:33:19.486236+0000","last_peered":"2024-09-12T01:33:19.486236+0000","last_clean":"2024-09-12T01:33:19.486236+0000","last_became_active":"2024-09-12T01:33:11.141976+0000","last_became_peered":"2024-09-12T01:33:11.141976+0000","last_unstale":"2024-09-12T01:33:19.486236+0000","last_undegraded":"2024-09-12T01:33:19.486236+0000","last_fullsized":"2024-09-12T01:33:19.486236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:11.527363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485800+0000","last_change":"2024-09-12T01:33:16.200223+0000","last_active":"2024-09-12T01:33:19.485800+0000","last_peered":"2024-09-12T01:33:19.485800+0000","last_clean":"2024-09-12T01:33:19.485800+0000","last_became_active":"2024-09-12T01:33:16.199767+0000","last_became_peered":"2024-09-12T01:33:16.199767+0000","last_unstale":"2024-09-12T01:33:19.485800+0000","last_undegraded":"2024-09-12T01:33:19.485800+0000","last_fullsized":"2024-09-12T01:33:19.485800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:01.750852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218829+0000","last_change":"2024-09-12T01:33:12.171774+0000","last_active":"2024-09-12T01:33:18.218829+0000","last_peered":"2024-09-12T01:33:18.218829+0000","last_clean":"2024-09-12T01:33:18.218829+0000","last_became_active":"2024-09-12T01:33:12.171472+0000","last_became_peered":"2024-09-12T01:33:12.171472+0000","last_unstale":"2024-09-12T01:33:18.218829+0000","last_undegraded":"2024-09-12T01:33:18.218829+0000","last_fullsized":"2024-09-12T01:33:18.218829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:50.682111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839456+0000","last_change":"2024-09-12T01:33:11.146150+0000","last_active":"2024-09-12T01:33:18.839456+0000","last_peered":"2024-09-12T01:33:18.839456+0000","last_clean":"2024-09-12T01:33:18.839456+0000","last_became_active":"2024-09-12T01:33:11.145690+0000","last_became_peered":"2024-09-12T01:33:11.145690+0000","last_unstale":"2024-09-12T01:33:18.839456+0000","last_undegraded":"2024-09-12T01:33:18.839456+0000","last_fullsized":"2024-09-12T01:33:18.839456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:49.680053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486103+0000","last_change":"2024-09-12T01:33:16.200773+0000","last_active":"2024-09-12T01:33:19.486103+0000","last_peered":"2024-09-12T01:33:19.486103+0000","last_clean":"2024-09-12T01:33:19.486103+0000","last_became_active":"2024-09-12T01:33:16.199757+0000","last_became_peered":"2024-09-12T01:33:16.199757+0000","last_unstale":"2024-09-12T01:33:19.486103+0000","last_undegraded":"2024-09-12T01:33:19.486103+0000","last_fullsized":"2024-09-12T01:33:19.486103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:40.563121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485669+0000","last_change":"2024-09-12T01:33:12.162181+0000","last_active":"2024-09-12T01:33:19.485669+0000","last_peered":"2024-09-12T01:33:19.485669+0000","last_clean":"2024-09-12T01:33:19.485669+0000","last_became_active":"2024-09-12T01:33:12.161900+0000","last_became_peered":"2024-09-12T01:33:12.161900+0000","last_unstale":"2024-09-12T01:33:19.485669+0000","last_undegraded":"2024-09-12T01:33:19.485669+0000","last_fullsized":"2024-09-12T01:33:19.485669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:52.439908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554120+0000","last_change":"2024-09-12T01:33:11.147499+0000","last_active":"2024-09-12T01:33:18.554120+0000","last_peered":"2024-09-12T01:33:18.554120+0000","last_clean":"2024-09-12T01:33:18.554120+0000","last_became_active":"2024-09-12T01:33:11.147186+0000","last_became_peered":"2024-09-12T01:33:11.147186+0000","last_unstale":"2024-09-12T01:33:18.554120+0000","last_undegraded":"2024-09-12T01:33:18.554120+0000","last_fullsized":"2024-09-12T01:33:18.554120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:15.921424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486454+0000","last_change":"2024-09-12T01:33:16.201262+0000","last_active":"2024-09-12T01:33:19.486454+0000","last_peered":"2024-09-12T01:33:19.486454+0000","last_clean":"2024-09-12T01:33:19.486454+0000","last_became_active":"2024-09-12T01:33:16.200850+0000","last_became_peered":"2024-09-12T01:33:16.200850+0000","last_unstale":"2024-09-12T01:33:19.486454+0000","last_undegraded":"2024-09-12T01:33:19.486454+0000","last_fullsized":"2024-09-12T01:33:19.486454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:13.593524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.412948+0000","last_change":"2024-09-12T01:33:12.177601+0000","last_active":"2024-09-12T01:33:19.412948+0000","last_peered":"2024-09-12T01:33:19.412948+0000","last_clean":"2024-09-12T01:33:19.412948+0000","last_became_active":"2024-09-12T01:33:12.177310+0000","last_became_peered":"2024-09-12T01:33:12.177310+0000","last_unstale":"2024-09-12T01:33:19.412948+0000","last_undegraded":"2024-09-12T01:33:19.412948+0000","last_fullsized":"2024-09-12T01:33:19.412948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:59.541178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486689+0000","last_change":"2024-09-12T01:33:11.151233+0000","last_active":"2024-09-12T01:33:19.486689+0000","last_peered":"2024-09-12T01:33:19.486689+0000","last_clean":"2024-09-12T01:33:19.486689+0000","last_became_active":"2024-09-12T01:33:11.150960+0000","last_became_peered":"2024-09-12T01:33:11.150960+0000","last_unstale":"2024-09-12T01:33:19.486689+0000","last_undegraded":"2024-09-12T01:33:19.486689+0000","last_fullsized":"2024-09-12T01:33:19.486689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413558+0000","last_change":"2024-09-12T01:33:16.593340+0000","last_active":"2024-09-12T01:33:19.413558+0000","last_peered":"2024-09-12T01:33:19.413558+0000","last_clean":"2024-09-12T01:33:19.413558+0000","last_became_active":"2024-09-12T01:33:16.592942+0000","last_became_peered":"2024-09-12T01:33:16.592942+0000","last_unstale":"2024-09-12T01:33:19.413558+0000","last_undegraded":"2024-09-12T01:33:19.413558+0000","last_fullsized":"2024-09-12T01:33:19.413558+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:06.180609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216813+0000","last_change":"2024-09-12T01:33:12.175958+0000","last_active":"2024-09-12T01:33:18.216813+0000","last_peered":"2024-09-12T01:33:18.216813+0000","last_clean":"2024-09-12T01:33:18.216813+0000","last_became_active":"2024-09-12T01:33:12.174098+0000","last_became_peered":"2024-09-12T01:33:12.174098+0000","last_unstale":"2024-09-12T01:33:18.216813+0000","last_undegraded":"2024-09-12T01:33:18.216813+0000","last_fullsized":"2024-09-12T01:33:18.216813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839416+0000","last_change":"2024-09-12T01:33:11.148301+0000","last_active":"2024-09-12T01:33:18.839416+0000","last_peered":"2024-09-12T01:33:18.839416+0000","last_clean":"2024-09-12T01:33:18.839416+0000","last_became_active":"2024-09-12T01:33:11.146483+0000","last_became_peered":"2024-09-12T01:33:11.146483+0000","last_unstale":"2024-09-12T01:33:18.839416+0000","last_undegraded":"2024-09-12T01:33:18.839416+0000","last_fullsized":"2024-09-12T01:33:18.839416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:11.527363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486566+0000","last_change":"2024-09-12T01:33:01.096542+0000","last_active":"2024-09-12T01:33:19.486566+0000","last_peered":"2024-09-12T01:33:19.486566+0000","last_clean":"2024-09-12T01:33:19.486566+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:33:19.486566+0000","last_undegraded":"2024-09-12T01:33:19.486566+0000","last_fullsized":"2024-09-12T01:33:19.486566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_clean_scrub_stamp":"2024-09-12T01:32:58.512553+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:48:58.741813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486931+0000","last_change":"2024-09-12T01:33:16.591360+0000","last_active":"2024-09-12T01:33:19.486931+0000","last_peered":"2024-09-12T01:33:19.486931+0000","last_clean":"2024-09-12T01:33:19.486931+0000","last_became_active":"2024-09-12T01:33:16.590884+0000","last_became_peered":"2024-09-12T01:33:16.590884+0000","last_unstale":"2024-09-12T01:33:19.486931+0000","last_undegraded":"2024-09-12T01:33:19.486931+0000","last_fullsized":"2024-09-12T01:33:19.486931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:15.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487399+0000","last_change":"2024-09-12T01:33:12.178384+0000","last_active":"2024-09-12T01:33:19.487399+0000","last_peered":"2024-09-12T01:33:19.487399+0000","last_clean":"2024-09-12T01:33:19.487399+0000","last_became_active":"2024-09-12T01:33:12.178068+0000","last_became_peered":"2024-09-12T01:33:12.178068+0000","last_unstale":"2024-09-12T01:33:19.487399+0000","last_undegraded":"2024-09-12T01:33:19.487399+0000","last_fullsized":"2024-09-12T01:33:19.487399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:51.254891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554226+0000","last_change":"2024-09-12T01:33:11.149796+0000","last_active":"2024-09-12T01:33:18.554226+0000","last_peered":"2024-09-12T01:33:18.554226+0000","last_clean":"2024-09-12T01:33:18.554226+0000","last_became_active":"2024-09-12T01:33:11.149505+0000","last_became_peered":"2024-09-12T01:33:11.149505+0000","last_unstale":"2024-09-12T01:33:18.554226+0000","last_undegraded":"2024-09-12T01:33:18.554226+0000","last_fullsized":"2024-09-12T01:33:18.554226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:24.957500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486471+0000","last_change":"2024-09-12T01:33:16.589125+0000","last_active":"2024-09-12T01:33:19.486471+0000","last_peered":"2024-09-12T01:33:19.486471+0000","last_clean":"2024-09-12T01:33:19.486471+0000","last_became_active":"2024-09-12T01:33:16.588637+0000","last_became_peered":"2024-09-12T01:33:16.588637+0000","last_unstale":"2024-09-12T01:33:19.486471+0000","last_undegraded":"2024-09-12T01:33:19.486471+0000","last_fullsized":"2024-09-12T01:33:19.486471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:06.997893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839688+0000","last_change":"2024-09-12T01:33:12.176967+0000","last_active":"2024-09-12T01:33:18.839688+0000","last_peered":"2024-09-12T01:33:18.839688+0000","last_clean":"2024-09-12T01:33:18.839688+0000","last_became_active":"2024-09-12T01:33:12.176355+0000","last_became_peered":"2024-09-12T01:33:12.176355+0000","last_unstale":"2024-09-12T01:33:18.839688+0000","last_undegraded":"2024-09-12T01:33:18.839688+0000","last_fullsized":"2024-09-12T01:33:18.839688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:57.532127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485317+0000","last_change":"2024-09-12T01:33:11.149279+0000","last_active":"2024-09-12T01:33:19.485317+0000","last_peered":"2024-09-12T01:33:19.485317+0000","last_clean":"2024-09-12T01:33:19.485317+0000","last_became_active":"2024-09-12T01:33:11.148215+0000","last_became_peered":"2024-09-12T01:33:11.148215+0000","last_unstale":"2024-09-12T01:33:19.485317+0000","last_undegraded":"2024-09-12T01:33:19.485317+0000","last_fullsized":"2024-09-12T01:33:19.485317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:49.680053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413355+0000","last_change":"2024-09-12T01:33:16.200672+0000","last_active":"2024-09-12T01:33:19.413355+0000","last_peered":"2024-09-12T01:33:19.413355+0000","last_clean":"2024-09-12T01:33:19.413355+0000","last_became_active":"2024-09-12T01:33:16.200251+0000","last_became_peered":"2024-09-12T01:33:16.200251+0000","last_unstale":"2024-09-12T01:33:19.413355+0000","last_undegraded":"2024-09-12T01:33:19.413355+0000","last_fullsized":"2024-09-12T01:33:19.413355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:37.500171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487810+0000","last_change":"2024-09-12T01:33:12.179113+0000","last_active":"2024-09-12T01:33:19.487810+0000","last_peered":"2024-09-12T01:33:19.487810+0000","last_clean":"2024-09-12T01:33:19.487810+0000","last_became_active":"2024-09-12T01:33:12.178495+0000","last_became_peered":"2024-09-12T01:33:12.178495+0000","last_unstale":"2024-09-12T01:33:19.487810+0000","last_undegraded":"2024-09-12T01:33:19.487810+0000","last_fullsized":"2024-09-12T01:33:19.487810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:51:57.703184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554326+0000","last_change":"2024-09-12T01:33:11.144585+0000","last_active":"2024-09-12T01:33:18.554326+0000","last_peered":"2024-09-12T01:33:18.554326+0000","last_clean":"2024-09-12T01:33:18.554326+0000","last_became_active":"2024-09-12T01:33:11.144277+0000","last_became_peered":"2024-09-12T01:33:11.144277+0000","last_unstale":"2024-09-12T01:33:18.554326+0000","last_undegraded":"2024-09-12T01:33:18.554326+0000","last_fullsized":"2024-09-12T01:33:18.554326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:15.342457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839029+0000","last_change":"2024-09-12T01:33:16.591894+0000","last_active":"2024-09-12T01:33:18.839029+0000","last_peered":"2024-09-12T01:33:18.839029+0000","last_clean":"2024-09-12T01:33:18.839029+0000","last_became_active":"2024-09-12T01:33:16.591452+0000","last_became_peered":"2024-09-12T01:33:16.591452+0000","last_unstale":"2024-09-12T01:33:18.839029+0000","last_undegraded":"2024-09-12T01:33:18.839029+0000","last_fullsized":"2024-09-12T01:33:18.839029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:59.574294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487365+0000","last_change":"2024-09-12T01:33:12.176012+0000","last_active":"2024-09-12T01:33:19.487365+0000","last_peered":"2024-09-12T01:33:19.487365+0000","last_clean":"2024-09-12T01:33:19.487365+0000","last_became_active":"2024-09-12T01:33:12.175729+0000","last_became_peered":"2024-09-12T01:33:12.175729+0000","last_unstale":"2024-09-12T01:33:19.487365+0000","last_undegraded":"2024-09-12T01:33:19.487365+0000","last_fullsized":"2024-09-12T01:33:19.487365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:57.532127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.219006+0000","last_change":"2024-09-12T01:33:11.138837+0000","last_active":"2024-09-12T01:33:18.219006+0000","last_peered":"2024-09-12T01:33:18.219006+0000","last_clean":"2024-09-12T01:33:18.219006+0000","last_became_active":"2024-09-12T01:33:11.138324+0000","last_became_peered":"2024-09-12T01:33:11.138324+0000","last_unstale":"2024-09-12T01:33:18.219006+0000","last_undegraded":"2024-09-12T01:33:18.219006+0000","last_fullsized":"2024-09-12T01:33:18.219006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:48.706430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217732+0000","last_change":"2024-09-12T01:33:16.200428+0000","last_active":"2024-09-12T01:33:18.217732+0000","last_peered":"2024-09-12T01:33:18.217732+0000","last_clean":"2024-09-12T01:33:18.217732+0000","last_became_active":"2024-09-12T01:33:16.199943+0000","last_became_peered":"2024-09-12T01:33:16.199943+0000","last_unstale":"2024-09-12T01:33:18.217732+0000","last_undegraded":"2024-09-12T01:33:18.217732+0000","last_fullsized":"2024-09-12T01:33:18.217732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:15.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486512+0000","last_change":"2024-09-12T01:33:12.176546+0000","last_active":"2024-09-12T01:33:19.486512+0000","last_peered":"2024-09-12T01:33:19.486512+0000","last_clean":"2024-09-12T01:33:19.486512+0000","last_became_active":"2024-09-12T01:33:12.176254+0000","last_became_peered":"2024-09-12T01:33:12.176254+0000","last_unstale":"2024-09-12T01:33:19.486512+0000","last_undegraded":"2024-09-12T01:33:19.486512+0000","last_fullsized":"2024-09-12T01:33:19.486512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:24.363602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.553875+0000","last_change":"2024-09-12T01:33:11.143423+0000","last_active":"2024-09-12T01:33:18.553875+0000","last_peered":"2024-09-12T01:33:18.553875+0000","last_clean":"2024-09-12T01:33:18.553875+0000","last_became_active":"2024-09-12T01:33:11.143125+0000","last_became_peered":"2024-09-12T01:33:11.143125+0000","last_unstale":"2024-09-12T01:33:18.553875+0000","last_undegraded":"2024-09-12T01:33:18.553875+0000","last_fullsized":"2024-09-12T01:33:18.553875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:24.005795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413405+0000","last_change":"2024-09-12T01:33:16.190124+0000","last_active":"2024-09-12T01:33:19.413405+0000","last_peered":"2024-09-12T01:33:19.413405+0000","last_clean":"2024-09-12T01:33:19.413405+0000","last_became_active":"2024-09-12T01:33:16.189728+0000","last_became_peered":"2024-09-12T01:33:16.189728+0000","last_unstale":"2024-09-12T01:33:19.413405+0000","last_undegraded":"2024-09-12T01:33:19.413405+0000","last_fullsized":"2024-09-12T01:33:19.413405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:00.559557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486163+0000","last_change":"2024-09-12T01:33:12.173415+0000","last_active":"2024-09-12T01:33:19.486163+0000","last_peered":"2024-09-12T01:33:19.486163+0000","last_clean":"2024-09-12T01:33:19.486163+0000","last_became_active":"2024-09-12T01:33:12.173117+0000","last_became_peered":"2024-09-12T01:33:12.173117+0000","last_unstale":"2024-09-12T01:33:19.486163+0000","last_undegraded":"2024-09-12T01:33:19.486163+0000","last_fullsized":"2024-09-12T01:33:19.486163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:54.679349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486025+0000","last_change":"2024-09-12T01:33:11.140718+0000","last_active":"2024-09-12T01:33:19.486025+0000","last_peered":"2024-09-12T01:33:19.486025+0000","last_clean":"2024-09-12T01:33:19.486025+0000","last_became_active":"2024-09-12T01:33:11.140321+0000","last_became_peered":"2024-09-12T01:33:11.140321+0000","last_unstale":"2024-09-12T01:33:19.486025+0000","last_undegraded":"2024-09-12T01:33:19.486025+0000","last_fullsized":"2024-09-12T01:33:19.486025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:22.885518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217295+0000","last_change":"2024-09-12T01:33:16.594875+0000","last_active":"2024-09-12T01:33:18.217295+0000","last_peered":"2024-09-12T01:33:18.217295+0000","last_clean":"2024-09-12T01:33:18.217295+0000","last_became_active":"2024-09-12T01:33:16.592733+0000","last_became_peered":"2024-09-12T01:33:16.592733+0000","last_unstale":"2024-09-12T01:33:18.217295+0000","last_undegraded":"2024-09-12T01:33:18.217295+0000","last_fullsized":"2024-09-12T01:33:18.217295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:39.080379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486853+0000","last_change":"2024-09-12T01:33:12.175983+0000","last_active":"2024-09-12T01:33:19.486853+0000","last_peered":"2024-09-12T01:33:19.486853+0000","last_clean":"2024-09-12T01:33:19.486853+0000","last_became_active":"2024-09-12T01:33:12.175638+0000","last_became_peered":"2024-09-12T01:33:12.175638+0000","last_unstale":"2024-09-12T01:33:19.486853+0000","last_undegraded":"2024-09-12T01:33:19.486853+0000","last_fullsized":"2024-09-12T01:33:19.486853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:48.292206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839432+0000","last_change":"2024-09-12T01:33:11.148832+0000","last_active":"2024-09-12T01:33:18.839432+0000","last_peered":"2024-09-12T01:33:18.839432+0000","last_clean":"2024-09-12T01:33:18.839432+0000","last_became_active":"2024-09-12T01:33:11.148519+0000","last_became_peered":"2024-09-12T01:33:11.148519+0000","last_unstale":"2024-09-12T01:33:18.839432+0000","last_undegraded":"2024-09-12T01:33:18.839432+0000","last_fullsized":"2024-09-12T01:33:18.839432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217452+0000","last_change":"2024-09-12T01:33:16.588958+0000","last_active":"2024-09-12T01:33:18.217452+0000","last_peered":"2024-09-12T01:33:18.217452+0000","last_clean":"2024-09-12T01:33:18.217452+0000","last_became_active":"2024-09-12T01:33:16.588510+0000","last_became_peered":"2024-09-12T01:33:16.588510+0000","last_unstale":"2024-09-12T01:33:18.217452+0000","last_undegraded":"2024-09-12T01:33:18.217452+0000","last_fullsized":"2024-09-12T01:33:18.217452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:00.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413221+0000","last_change":"2024-09-12T01:33:12.184393+0000","last_active":"2024-09-12T01:33:19.413221+0000","last_peered":"2024-09-12T01:33:19.413221+0000","last_clean":"2024-09-12T01:33:19.413221+0000","last_became_active":"2024-09-12T01:33:12.184103+0000","last_became_peered":"2024-09-12T01:33:12.184103+0000","last_unstale":"2024-09-12T01:33:19.413221+0000","last_undegraded":"2024-09-12T01:33:19.413221+0000","last_fullsized":"2024-09-12T01:33:19.413221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:36.026056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.412683+0000","last_change":"2024-09-12T01:33:11.134020+0000","last_active":"2024-09-12T01:33:19.412683+0000","last_peered":"2024-09-12T01:33:19.412683+0000","last_clean":"2024-09-12T01:33:19.412683+0000","last_became_active":"2024-09-12T01:33:11.133717+0000","last_became_peered":"2024-09-12T01:33:11.133717+0000","last_unstale":"2024-09-12T01:33:19.412683+0000","last_undegraded":"2024-09-12T01:33:19.412683+0000","last_fullsized":"2024-09-12T01:33:19.412683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:20.901290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839735+0000","last_change":"2024-09-12T01:33:16.593051+0000","last_active":"2024-09-12T01:33:18.839735+0000","last_peered":"2024-09-12T01:33:18.839735+0000","last_clean":"2024-09-12T01:33:18.839735+0000","last_became_active":"2024-09-12T01:33:16.592513+0000","last_became_peered":"2024-09-12T01:33:16.592513+0000","last_unstale":"2024-09-12T01:33:18.839735+0000","last_undegraded":"2024-09-12T01:33:18.839735+0000","last_fullsized":"2024-09-12T01:33:18.839735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:35.903448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487266+0000","last_change":"2024-09-12T01:33:12.178596+0000","last_active":"2024-09-12T01:33:19.487266+0000","last_peered":"2024-09-12T01:33:19.487266+0000","last_clean":"2024-09-12T01:33:19.487266+0000","last_became_active":"2024-09-12T01:33:12.176316+0000","last_became_peered":"2024-09-12T01:33:12.176316+0000","last_unstale":"2024-09-12T01:33:19.487266+0000","last_undegraded":"2024-09-12T01:33:19.487266+0000","last_fullsized":"2024-09-12T01:33:19.487266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:24.134321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:21.709458+0000","last_change":"2024-09-12T01:33:11.147275+0000","last_active":"2024-09-12T01:33:21.709458+0000","last_peered":"2024-09-12T01:33:21.709458+0000","last_clean":"2024-09-12T01:33:21.709458+0000","last_became_active":"2024-09-12T01:33:11.146962+0000","last_became_peered":"2024-09-12T01:33:11.146962+0000","last_unstale":"2024-09-12T01:33:21.709458+0000","last_undegraded":"2024-09-12T01:33:21.709458+0000","last_fullsized":"2024-09-12T01:33:21.709458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:11.527363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485898+0000","last_change":"2024-09-12T01:33:16.202736+0000","last_active":"2024-09-12T01:33:19.485898+0000","last_peered":"2024-09-12T01:33:19.485898+0000","last_clean":"2024-09-12T01:33:19.485898+0000","last_became_active":"2024-09-12T01:33:16.202282+0000","last_became_peered":"2024-09-12T01:33:16.202282+0000","last_unstale":"2024-09-12T01:33:19.485898+0000","last_undegraded":"2024-09-12T01:33:19.485898+0000","last_fullsized":"2024-09-12T01:33:19.485898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:28.321407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218580+0000","last_change":"2024-09-12T01:33:12.173488+0000","last_active":"2024-09-12T01:33:18.218580+0000","last_peered":"2024-09-12T01:33:18.218580+0000","last_clean":"2024-09-12T01:33:18.218580+0000","last_became_active":"2024-09-12T01:33:12.173187+0000","last_became_peered":"2024-09-12T01:33:12.173187+0000","last_unstale":"2024-09-12T01:33:18.218580+0000","last_undegraded":"2024-09-12T01:33:18.218580+0000","last_fullsized":"2024-09-12T01:33:18.218580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:27.581552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485330+0000","last_change":"2024-09-12T01:33:11.149565+0000","last_active":"2024-09-12T01:33:19.485330+0000","last_peered":"2024-09-12T01:33:19.485330+0000","last_clean":"2024-09-12T01:33:19.485330+0000","last_became_active":"2024-09-12T01:33:11.149034+0000","last_became_peered":"2024-09-12T01:33:11.149034+0000","last_unstale":"2024-09-12T01:33:19.485330+0000","last_undegraded":"2024-09-12T01:33:19.485330+0000","last_fullsized":"2024-09-12T01:33:19.485330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218212+0000","last_change":"2024-09-12T01:33:16.202414+0000","last_active":"2024-09-12T01:33:18.218212+0000","last_peered":"2024-09-12T01:33:18.218212+0000","last_clean":"2024-09-12T01:33:18.218212+0000","last_became_active":"2024-09-12T01:33:16.200143+0000","last_became_peered":"2024-09-12T01:33:16.200143+0000","last_unstale":"2024-09-12T01:33:18.218212+0000","last_undegraded":"2024-09-12T01:33:18.218212+0000","last_fullsized":"2024-09-12T01:33:18.218212+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:14.088579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413595+0000","last_change":"2024-09-12T01:33:12.174901+0000","last_active":"2024-09-12T01:33:19.413595+0000","last_peered":"2024-09-12T01:33:19.413595+0000","last_clean":"2024-09-12T01:33:19.413595+0000","last_became_active":"2024-09-12T01:33:12.174638+0000","last_became_peered":"2024-09-12T01:33:12.174638+0000","last_unstale":"2024-09-12T01:33:19.413595+0000","last_undegraded":"2024-09-12T01:33:19.413595+0000","last_fullsized":"2024-09-12T01:33:19.413595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:54.169299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839752+0000","last_change":"2024-09-12T01:33:11.147387+0000","last_active":"2024-09-12T01:33:18.839752+0000","last_peered":"2024-09-12T01:33:18.839752+0000","last_clean":"2024-09-12T01:33:18.839752+0000","last_became_active":"2024-09-12T01:33:11.147110+0000","last_became_peered":"2024-09-12T01:33:11.147110+0000","last_unstale":"2024-09-12T01:33:18.839752+0000","last_undegraded":"2024-09-12T01:33:18.839752+0000","last_fullsized":"2024-09-12T01:33:18.839752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216657+0000","last_change":"2024-09-12T01:33:16.191800+0000","last_active":"2024-09-12T01:33:18.216657+0000","last_peered":"2024-09-12T01:33:18.216657+0000","last_clean":"2024-09-12T01:33:18.216657+0000","last_became_active":"2024-09-12T01:33:16.190448+0000","last_became_peered":"2024-09-12T01:33:16.190448+0000","last_unstale":"2024-09-12T01:33:18.216657+0000","last_undegraded":"2024-09-12T01:33:18.216657+0000","last_fullsized":"2024-09-12T01:33:18.216657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:47.099767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839819+0000","last_change":"2024-09-12T01:33:12.178000+0000","last_active":"2024-09-12T01:33:18.839819+0000","last_peered":"2024-09-12T01:33:18.839819+0000","last_clean":"2024-09-12T01:33:18.839819+0000","last_became_active":"2024-09-12T01:33:12.177703+0000","last_became_peered":"2024-09-12T01:33:12.177703+0000","last_unstale":"2024-09-12T01:33:18.839819+0000","last_undegraded":"2024-09-12T01:33:18.839819+0000","last_fullsized":"2024-09-12T01:33:18.839819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:56.959747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485907+0000","last_change":"2024-09-12T01:33:11.149688+0000","last_active":"2024-09-12T01:33:19.485907+0000","last_peered":"2024-09-12T01:33:19.485907+0000","last_clean":"2024-09-12T01:33:19.485907+0000","last_became_active":"2024-09-12T01:33:11.149363+0000","last_became_peered":"2024-09-12T01:33:11.149363+0000","last_unstale":"2024-09-12T01:33:19.485907+0000","last_undegraded":"2024-09-12T01:33:19.485907+0000","last_fullsized":"2024-09-12T01:33:19.485907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216625+0000","last_change":"2024-09-12T01:33:16.592951+0000","last_active":"2024-09-12T01:33:18.216625+0000","last_peered":"2024-09-12T01:33:18.216625+0000","last_clean":"2024-09-12T01:33:18.216625+0000","last_became_active":"2024-09-12T01:33:16.592558+0000","last_became_peered":"2024-09-12T01:33:16.592558+0000","last_unstale":"2024-09-12T01:33:18.216625+0000","last_undegraded":"2024-09-12T01:33:18.216625+0000","last_fullsized":"2024-09-12T01:33:18.216625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:29.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413385+0000","last_change":"2024-09-12T01:33:16.594318+0000","last_active":"2024-09-12T01:33:19.413385+0000","last_peered":"2024-09-12T01:33:19.413385+0000","last_clean":"2024-09-12T01:33:19.413385+0000","last_became_active":"2024-09-12T01:33:16.593834+0000","last_became_peered":"2024-09-12T01:33:16.593834+0000","last_unstale":"2024-09-12T01:33:19.413385+0000","last_undegraded":"2024-09-12T01:33:19.413385+0000","last_fullsized":"2024-09-12T01:33:19.413385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:11.126504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485462+0000","last_change":"2024-09-12T01:33:12.171282+0000","last_active":"2024-09-12T01:33:19.485462+0000","last_peered":"2024-09-12T01:33:19.485462+0000","last_clean":"2024-09-12T01:33:19.485462+0000","last_became_active":"2024-09-12T01:33:12.170988+0000","last_became_peered":"2024-09-12T01:33:12.170988+0000","last_unstale":"2024-09-12T01:33:19.485462+0000","last_undegraded":"2024-09-12T01:33:19.485462+0000","last_fullsized":"2024-09-12T01:33:19.485462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:25.007983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839177+0000","last_change":"2024-09-12T01:33:11.146497+0000","last_active":"2024-09-12T01:33:18.839177+0000","last_peered":"2024-09-12T01:33:18.839177+0000","last_clean":"2024-09-12T01:33:18.839177+0000","last_became_active":"2024-09-12T01:33:11.146184+0000","last_became_peered":"2024-09-12T01:33:11.146184+0000","last_unstale":"2024-09-12T01:33:18.839177+0000","last_undegraded":"2024-09-12T01:33:18.839177+0000","last_fullsized":"2024-09-12T01:33:18.839177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487513+0000","last_change":"2024-09-12T01:33:16.202584+0000","last_active":"2024-09-12T01:33:19.487513+0000","last_peered":"2024-09-12T01:33:19.487513+0000","last_clean":"2024-09-12T01:33:19.487513+0000","last_became_active":"2024-09-12T01:33:16.202137+0000","last_became_peered":"2024-09-12T01:33:16.202137+0000","last_unstale":"2024-09-12T01:33:19.487513+0000","last_undegraded":"2024-09-12T01:33:19.487513+0000","last_fullsized":"2024-09-12T01:33:19.487513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:39.155565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216613+0000","last_change":"2024-09-12T01:33:12.169183+0000","last_active":"2024-09-12T01:33:18.216613+0000","last_peered":"2024-09-12T01:33:18.216613+0000","last_clean":"2024-09-12T01:33:18.216613+0000","last_became_active":"2024-09-12T01:33:12.168803+0000","last_became_peered":"2024-09-12T01:33:12.168803+0000","last_unstale":"2024-09-12T01:33:18.216613+0000","last_undegraded":"2024-09-12T01:33:18.216613+0000","last_fullsized":"2024-09-12T01:33:18.216613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:25.007983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216934+0000","last_change":"2024-09-12T01:33:11.144264+0000","last_active":"2024-09-12T01:33:18.216934+0000","last_peered":"2024-09-12T01:33:18.216934+0000","last_clean":"2024-09-12T01:33:18.216934+0000","last_became_active":"2024-09-12T01:33:11.143993+0000","last_became_peered":"2024-09-12T01:33:11.143993+0000","last_unstale":"2024-09-12T01:33:18.216934+0000","last_undegraded":"2024-09-12T01:33:18.216934+0000","last_fullsized":"2024-09-12T01:33:18.216934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:02:33.335627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485661+0000","last_change":"2024-09-12T01:33:11.143410+0000","last_active":"2024-09-12T01:33:19.485661+0000","last_peered":"2024-09-12T01:33:19.485661+0000","last_clean":"2024-09-12T01:33:19.485661+0000","last_became_active":"2024-09-12T01:33:11.143121+0000","last_became_peered":"2024-09-12T01:33:11.143121+0000","last_unstale":"2024-09-12T01:33:19.485661+0000","last_undegraded":"2024-09-12T01:33:19.485661+0000","last_fullsized":"2024-09-12T01:33:19.485661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413493+0000","last_change":"2024-09-12T01:33:12.176891+0000","last_active":"2024-09-12T01:33:19.413493+0000","last_peered":"2024-09-12T01:33:19.413493+0000","last_clean":"2024-09-12T01:33:19.413493+0000","last_became_active":"2024-09-12T01:33:12.176573+0000","last_became_peered":"2024-09-12T01:33:12.176573+0000","last_unstale":"2024-09-12T01:33:19.413493+0000","last_undegraded":"2024-09-12T01:33:19.413493+0000","last_fullsized":"2024-09-12T01:33:19.413493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:47.157354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554126+0000","last_change":"2024-09-12T01:33:16.195123+0000","last_active":"2024-09-12T01:33:18.554126+0000","last_peered":"2024-09-12T01:33:18.554126+0000","last_clean":"2024-09-12T01:33:18.554126+0000","last_became_active":"2024-09-12T01:33:16.194690+0000","last_became_peered":"2024-09-12T01:33:16.194690+0000","last_unstale":"2024-09-12T01:33:18.554126+0000","last_undegraded":"2024-09-12T01:33:18.554126+0000","last_fullsized":"2024-09-12T01:33:18.554126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:14.296077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217027+0000","last_change":"2024-09-12T01:33:11.145695+0000","last_active":"2024-09-12T01:33:18.217027+0000","last_peered":"2024-09-12T01:33:18.217027+0000","last_clean":"2024-09-12T01:33:18.217027+0000","last_became_active":"2024-09-12T01:33:11.145391+0000","last_became_peered":"2024-09-12T01:33:11.145391+0000","last_unstale":"2024-09-12T01:33:18.217027+0000","last_undegraded":"2024-09-12T01:33:18.217027+0000","last_fullsized":"2024-09-12T01:33:18.217027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:47.695596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218383+0000","last_change":"2024-09-12T01:33:12.164156+0000","last_active":"2024-09-12T01:33:18.218383+0000","last_peered":"2024-09-12T01:33:18.218383+0000","last_clean":"2024-09-12T01:33:18.218383+0000","last_became_active":"2024-09-12T01:33:12.160798+0000","last_became_peered":"2024-09-12T01:33:12.160798+0000","last_unstale":"2024-09-12T01:33:18.218383+0000","last_undegraded":"2024-09-12T01:33:18.218383+0000","last_fullsized":"2024-09-12T01:33:18.218383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:56.394003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217262+0000","last_change":"2024-09-12T01:33:16.198377+0000","last_active":"2024-09-12T01:33:18.217262+0000","last_peered":"2024-09-12T01:33:18.217262+0000","last_clean":"2024-09-12T01:33:18.217262+0000","last_became_active":"2024-09-12T01:33:16.197913+0000","last_became_peered":"2024-09-12T01:33:16.197913+0000","last_unstale":"2024-09-12T01:33:18.217262+0000","last_undegraded":"2024-09-12T01:33:18.217262+0000","last_fullsized":"2024-09-12T01:33:18.217262+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:56:05.124457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486041+0000","last_change":"2024-09-12T01:33:12.176137+0000","last_active":"2024-09-12T01:33:19.486041+0000","last_peered":"2024-09-12T01:33:19.486041+0000","last_clean":"2024-09-12T01:33:19.486041+0000","last_became_active":"2024-09-12T01:33:12.175840+0000","last_became_peered":"2024-09-12T01:33:12.175840+0000","last_unstale":"2024-09-12T01:33:19.486041+0000","last_undegraded":"2024-09-12T01:33:19.486041+0000","last_fullsized":"2024-09-12T01:33:19.486041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:38.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554012+0000","last_change":"2024-09-12T01:33:12.168134+0000","last_active":"2024-09-12T01:33:18.554012+0000","last_peered":"2024-09-12T01:33:18.554012+0000","last_clean":"2024-09-12T01:33:18.554012+0000","last_became_active":"2024-09-12T01:33:12.167844+0000","last_became_peered":"2024-09-12T01:33:12.167844+0000","last_unstale":"2024-09-12T01:33:18.554012+0000","last_undegraded":"2024-09-12T01:33:18.554012+0000","last_fullsized":"2024-09-12T01:33:18.554012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:06.175990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839339+0000","last_change":"2024-09-12T01:33:12.183112+0000","last_active":"2024-09-12T01:33:18.839339+0000","last_peered":"2024-09-12T01:33:18.839339+0000","last_clean":"2024-09-12T01:33:18.839339+0000","last_became_active":"2024-09-12T01:33:12.182858+0000","last_became_peered":"2024-09-12T01:33:12.182858+0000","last_unstale":"2024-09-12T01:33:18.839339+0000","last_undegraded":"2024-09-12T01:33:18.839339+0000","last_fullsized":"2024-09-12T01:33:18.839339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:11.508027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413461+0000","last_change":"2024-09-12T01:33:12.181683+0000","last_active":"2024-09-12T01:33:19.413461+0000","last_peered":"2024-09-12T01:33:19.413461+0000","last_clean":"2024-09-12T01:33:19.413461+0000","last_became_active":"2024-09-12T01:33:12.181383+0000","last_became_peered":"2024-09-12T01:33:12.181383+0000","last_unstale":"2024-09-12T01:33:19.413461+0000","last_undegraded":"2024-09-12T01:33:19.413461+0000","last_fullsized":"2024-09-12T01:33:19.413461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:00.865640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217578+0000","last_change":"2024-09-12T01:33:12.178479+0000","last_active":"2024-09-12T01:33:18.217578+0000","last_peered":"2024-09-12T01:33:18.217578+0000","last_clean":"2024-09-12T01:33:18.217578+0000","last_became_active":"2024-09-12T01:33:12.178198+0000","last_became_peered":"2024-09-12T01:33:12.178198+0000","last_unstale":"2024-09-12T01:33:18.217578+0000","last_undegraded":"2024-09-12T01:33:18.217578+0000","last_fullsized":"2024-09-12T01:33:18.217578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:53.398489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485933+0000","last_change":"2024-09-12T01:33:12.168261+0000","last_active":"2024-09-12T01:33:19.485933+0000","last_peered":"2024-09-12T01:33:19.485933+0000","last_clean":"2024-09-12T01:33:19.485933+0000","last_became_active":"2024-09-12T01:33:12.167980+0000","last_became_peered":"2024-09-12T01:33:12.167980+0000","last_unstale":"2024-09-12T01:33:19.485933+0000","last_undegraded":"2024-09-12T01:33:19.485933+0000","last_fullsized":"2024-09-12T01:33:19.485933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:02.393918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487377+0000","last_change":"2024-09-12T01:33:12.186333+0000","last_active":"2024-09-12T01:33:19.487377+0000","last_peered":"2024-09-12T01:33:19.487377+0000","last_clean":"2024-09-12T01:33:19.487377+0000","last_became_active":"2024-09-12T01:33:12.186047+0000","last_became_peered":"2024-09-12T01:33:12.186047+0000","last_unstale":"2024-09-12T01:33:19.487377+0000","last_undegraded":"2024-09-12T01:33:19.487377+0000","last_fullsized":"2024-09-12T01:33:19.487377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:53.398489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217203+0000","last_change":"2024-09-12T01:33:12.177911+0000","last_active":"2024-09-12T01:33:18.217203+0000","last_peered":"2024-09-12T01:33:18.217203+0000","last_clean":"2024-09-12T01:33:18.217203+0000","last_became_active":"2024-09-12T01:33:12.177625+0000","last_became_peered":"2024-09-12T01:33:12.177625+0000","last_unstale":"2024-09-12T01:33:18.217203+0000","last_undegraded":"2024-09-12T01:33:18.217203+0000","last_fullsized":"2024-09-12T01:33:18.217203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:02.393918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839173+0000","last_change":"2024-09-12T01:33:12.183935+0000","last_active":"2024-09-12T01:33:18.839173+0000","last_peered":"2024-09-12T01:33:18.839173+0000","last_clean":"2024-09-12T01:33:18.839173+0000","last_became_active":"2024-09-12T01:33:12.183620+0000","last_became_peered":"2024-09-12T01:33:12.183620+0000","last_unstale":"2024-09-12T01:33:18.839173+0000","last_undegraded":"2024-09-12T01:33:18.839173+0000","last_fullsized":"2024-09-12T01:33:18.839173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:02.950225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216744+0000","last_change":"2024-09-12T01:33:12.179130+0000","last_active":"2024-09-12T01:33:18.216744+0000","last_peered":"2024-09-12T01:33:18.216744+0000","last_clean":"2024-09-12T01:33:18.216744+0000","last_became_active":"2024-09-12T01:33:12.177141+0000","last_became_peered":"2024-09-12T01:33:12.177141+0000","last_unstale":"2024-09-12T01:33:18.216744+0000","last_undegraded":"2024-09-12T01:33:18.216744+0000","last_fullsized":"2024-09-12T01:33:18.216744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:24.134321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839306+0000","last_change":"2024-09-12T01:33:12.175899+0000","last_active":"2024-09-12T01:33:18.839306+0000","last_peered":"2024-09-12T01:33:18.839306+0000","last_clean":"2024-09-12T01:33:18.839306+0000","last_became_active":"2024-09-12T01:33:12.175594+0000","last_became_peered":"2024-09-12T01:33:12.175594+0000","last_unstale":"2024-09-12T01:33:18.839306+0000","last_undegraded":"2024-09-12T01:33:18.839306+0000","last_fullsized":"2024-09-12T01:33:18.839306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:51.254891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413194+0000","last_change":"2024-09-12T01:33:12.185737+0000","last_active":"2024-09-12T01:33:19.413194+0000","last_peered":"2024-09-12T01:33:19.413194+0000","last_clean":"2024-09-12T01:33:19.413194+0000","last_became_active":"2024-09-12T01:33:12.185489+0000","last_became_peered":"2024-09-12T01:33:12.185489+0000","last_unstale":"2024-09-12T01:33:19.413194+0000","last_undegraded":"2024-09-12T01:33:19.413194+0000","last_fullsized":"2024-09-12T01:33:19.413194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:05.301039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486993+0000","last_change":"2024-09-12T01:33:12.183324+0000","last_active":"2024-09-12T01:33:19.486993+0000","last_peered":"2024-09-12T01:33:19.486993+0000","last_clean":"2024-09-12T01:33:19.486993+0000","last_became_active":"2024-09-12T01:33:12.182857+0000","last_became_peered":"2024-09-12T01:33:12.182857+0000","last_unstale":"2024-09-12T01:33:19.486993+0000","last_undegraded":"2024-09-12T01:33:19.486993+0000","last_fullsized":"2024-09-12T01:33:19.486993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:52.439908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487522+0000","last_change":"2024-09-12T01:33:12.177539+0000","last_active":"2024-09-12T01:33:19.487522+0000","last_peered":"2024-09-12T01:33:19.487522+0000","last_clean":"2024-09-12T01:33:19.487522+0000","last_became_active":"2024-09-12T01:33:12.173980+0000","last_became_peered":"2024-09-12T01:33:12.173980+0000","last_unstale":"2024-09-12T01:33:19.487522+0000","last_undegraded":"2024-09-12T01:33:19.487522+0000","last_fullsized":"2024-09-12T01:33:19.487522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:11.508027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554334+0000","last_change":"2024-09-12T01:33:12.179198+0000","last_active":"2024-09-12T01:33:18.554334+0000","last_peered":"2024-09-12T01:33:18.554334+0000","last_clean":"2024-09-12T01:33:18.554334+0000","last_became_active":"2024-09-12T01:33:12.178859+0000","last_became_peered":"2024-09-12T01:33:12.178859+0000","last_unstale":"2024-09-12T01:33:18.554334+0000","last_undegraded":"2024-09-12T01:33:18.554334+0000","last_fullsized":"2024-09-12T01:33:18.554334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:46.228922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839997+0000","last_change":"2024-09-12T01:33:12.179001+0000","last_active":"2024-09-12T01:33:18.839997+0000","last_peered":"2024-09-12T01:33:18.839997+0000","last_clean":"2024-09-12T01:33:18.839997+0000","last_became_active":"2024-09-12T01:33:12.178298+0000","last_became_peered":"2024-09-12T01:33:12.178298+0000","last_unstale":"2024-09-12T01:33:18.839997+0000","last_undegraded":"2024-09-12T01:33:18.839997+0000","last_fullsized":"2024-09-12T01:33:18.839997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:35.107896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839003+0000","last_change":"2024-09-12T01:33:12.169863+0000","last_active":"2024-09-12T01:33:18.839003+0000","last_peered":"2024-09-12T01:33:18.839003+0000","last_clean":"2024-09-12T01:33:18.839003+0000","last_became_active":"2024-09-12T01:33:12.169441+0000","last_became_peered":"2024-09-12T01:33:12.169441+0000","last_unstale":"2024-09-12T01:33:18.839003+0000","last_undegraded":"2024-09-12T01:33:18.839003+0000","last_fullsized":"2024-09-12T01:33:18.839003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485524+0000","last_change":"2024-09-12T01:33:12.160854+0000","last_active":"2024-09-12T01:33:19.485524+0000","last_peered":"2024-09-12T01:33:19.485524+0000","last_clean":"2024-09-12T01:33:19.485524+0000","last_became_active":"2024-09-12T01:33:12.160512+0000","last_became_peered":"2024-09-12T01:33:12.160512+0000","last_unstale":"2024-09-12T01:33:19.485524+0000","last_undegraded":"2024-09-12T01:33:19.485524+0000","last_fullsized":"2024-09-12T01:33:19.485524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:38.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413165+0000","last_change":"2024-09-12T01:33:12.172031+0000","last_active":"2024-09-12T01:33:19.413165+0000","last_peered":"2024-09-12T01:33:19.413165+0000","last_clean":"2024-09-12T01:33:19.413165+0000","last_became_active":"2024-09-12T01:33:12.171730+0000","last_became_peered":"2024-09-12T01:33:12.171730+0000","last_unstale":"2024-09-12T01:33:19.413165+0000","last_undegraded":"2024-09-12T01:33:19.413165+0000","last_fullsized":"2024-09-12T01:33:19.413165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:10.248408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216814+0000","last_change":"2024-09-12T01:33:12.173718+0000","last_active":"2024-09-12T01:33:18.216814+0000","last_peered":"2024-09-12T01:33:18.216814+0000","last_clean":"2024-09-12T01:33:18.216814+0000","last_became_active":"2024-09-12T01:33:12.171379+0000","last_became_peered":"2024-09-12T01:33:12.171379+0000","last_unstale":"2024-09-12T01:33:18.216814+0000","last_undegraded":"2024-09-12T01:33:18.216814+0000","last_fullsized":"2024-09-12T01:33:18.216814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:52.439908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218338+0000","last_change":"2024-09-12T01:33:12.174895+0000","last_active":"2024-09-12T01:33:18.218338+0000","last_peered":"2024-09-12T01:33:18.218338+0000","last_clean":"2024-09-12T01:33:18.218338+0000","last_became_active":"2024-09-12T01:33:12.174591+0000","last_became_peered":"2024-09-12T01:33:12.174591+0000","last_unstale":"2024-09-12T01:33:18.218338+0000","last_undegraded":"2024-09-12T01:33:18.218338+0000","last_fullsized":"2024-09-12T01:33:18.218338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:57.102634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486963+0000","last_change":"2024-09-12T01:33:12.181588+0000","last_active":"2024-09-12T01:33:19.486963+0000","last_peered":"2024-09-12T01:33:19.486963+0000","last_clean":"2024-09-12T01:33:19.486963+0000","last_became_active":"2024-09-12T01:33:12.181293+0000","last_became_peered":"2024-09-12T01:33:12.181293+0000","last_unstale":"2024-09-12T01:33:19.486963+0000","last_undegraded":"2024-09-12T01:33:19.486963+0000","last_fullsized":"2024-09-12T01:33:19.486963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:02.393918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487308+0000","last_change":"2024-09-12T01:33:12.175792+0000","last_active":"2024-09-12T01:33:19.487308+0000","last_peered":"2024-09-12T01:33:19.487308+0000","last_clean":"2024-09-12T01:33:19.487308+0000","last_became_active":"2024-09-12T01:33:12.175332+0000","last_became_peered":"2024-09-12T01:33:12.175332+0000","last_unstale":"2024-09-12T01:33:19.487308+0000","last_undegraded":"2024-09-12T01:33:19.487308+0000","last_fullsized":"2024-09-12T01:33:19.487308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487796+0000","last_change":"2024-09-12T01:33:12.178831+0000","last_active":"2024-09-12T01:33:19.487796+0000","last_peered":"2024-09-12T01:33:19.487796+0000","last_clean":"2024-09-12T01:33:19.487796+0000","last_became_active":"2024-09-12T01:33:12.178116+0000","last_became_peered":"2024-09-12T01:33:12.178116+0000","last_unstale":"2024-09-12T01:33:19.487796+0000","last_undegraded":"2024-09-12T01:33:19.487796+0000","last_fullsized":"2024-09-12T01:33:19.487796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:35.107896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216514+0000","last_change":"2024-09-12T01:33:12.172225+0000","last_active":"2024-09-12T01:33:18.216514+0000","last_peered":"2024-09-12T01:33:18.216514+0000","last_clean":"2024-09-12T01:33:18.216514+0000","last_became_active":"2024-09-12T01:33:12.171933+0000","last_became_peered":"2024-09-12T01:33:12.171933+0000","last_unstale":"2024-09-12T01:33:18.216514+0000","last_undegraded":"2024-09-12T01:33:18.216514+0000","last_fullsized":"2024-09-12T01:33:18.216514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:20.867318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486602+0000","last_change":"2024-09-12T01:33:12.173395+0000","last_active":"2024-09-12T01:33:19.486602+0000","last_peered":"2024-09-12T01:33:19.486602+0000","last_clean":"2024-09-12T01:33:19.486602+0000","last_became_active":"2024-09-12T01:33:12.173090+0000","last_became_peered":"2024-09-12T01:33:12.173090+0000","last_unstale":"2024-09-12T01:33:19.486602+0000","last_undegraded":"2024-09-12T01:33:19.486602+0000","last_fullsized":"2024-09-12T01:33:19.486602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:46.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839213+0000","last_change":"2024-09-12T01:33:12.169604+0000","last_active":"2024-09-12T01:33:18.839213+0000","last_peered":"2024-09-12T01:33:18.839213+0000","last_clean":"2024-09-12T01:33:18.839213+0000","last_became_active":"2024-09-12T01:33:12.169258+0000","last_became_peered":"2024-09-12T01:33:12.169258+0000","last_unstale":"2024-09-12T01:33:18.839213+0000","last_undegraded":"2024-09-12T01:33:18.839213+0000","last_fullsized":"2024-09-12T01:33:18.839213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:46.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486571+0000","last_change":"2024-09-12T01:33:12.171033+0000","last_active":"2024-09-12T01:33:19.486571+0000","last_peered":"2024-09-12T01:33:19.486571+0000","last_clean":"2024-09-12T01:33:19.486571+0000","last_became_active":"2024-09-12T01:33:12.170708+0000","last_became_peered":"2024-09-12T01:33:12.170708+0000","last_unstale":"2024-09-12T01:33:19.486571+0000","last_undegraded":"2024-09-12T01:33:19.486571+0000","last_fullsized":"2024-09-12T01:33:19.486571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:56.959747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839539+0000","last_change":"2024-09-12T01:33:12.182780+0000","last_active":"2024-09-12T01:33:18.839539+0000","last_peered":"2024-09-12T01:33:18.839539+0000","last_clean":"2024-09-12T01:33:18.839539+0000","last_became_active":"2024-09-12T01:33:12.182454+0000","last_became_peered":"2024-09-12T01:33:12.182454+0000","last_unstale":"2024-09-12T01:33:18.839539+0000","last_undegraded":"2024-09-12T01:33:18.839539+0000","last_fullsized":"2024-09-12T01:33:18.839539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:53.398489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413295+0000","last_change":"2024-09-12T01:33:12.180723+0000","last_active":"2024-09-12T01:33:19.413295+0000","last_peered":"2024-09-12T01:33:19.413295+0000","last_clean":"2024-09-12T01:33:19.413295+0000","last_became_active":"2024-09-12T01:33:12.180414+0000","last_became_peered":"2024-09-12T01:33:12.180414+0000","last_unstale":"2024-09-12T01:33:19.413295+0000","last_undegraded":"2024-09-12T01:33:19.413295+0000","last_fullsized":"2024-09-12T01:33:19.413295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:58.901902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486128+0000","last_change":"2024-09-12T01:33:12.165745+0000","last_active":"2024-09-12T01:33:19.486128+0000","last_peered":"2024-09-12T01:33:19.486128+0000","last_clean":"2024-09-12T01:33:19.486128+0000","last_became_active":"2024-09-12T01:33:12.165475+0000","last_became_peered":"2024-09-12T01:33:12.165475+0000","last_unstale":"2024-09-12T01:33:19.486128+0000","last_undegraded":"2024-09-12T01:33:19.486128+0000","last_fullsized":"2024-09-12T01:33:19.486128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487685+0000","last_change":"2024-09-12T01:33:12.179105+0000","last_active":"2024-09-12T01:33:19.487685+0000","last_peered":"2024-09-12T01:33:19.487685+0000","last_clean":"2024-09-12T01:33:19.487685+0000","last_became_active":"2024-09-12T01:33:12.173779+0000","last_became_peered":"2024-09-12T01:33:12.173779+0000","last_unstale":"2024-09-12T01:33:19.487685+0000","last_undegraded":"2024-09-12T01:33:19.487685+0000","last_fullsized":"2024-09-12T01:33:19.487685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:02.950225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":700890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":113784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":110030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":110066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":282624,"data_stored":113784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":110100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":270336,"data_stored":109940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":876544,"data_stored":700946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:33:32.246 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:33:32.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.383+0000 7f8c6ae82640 1 Processor -- start 2024-09-12T01:33:32.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.383+0000 7f8c6ae82640 1 -- start start 2024-09-12T01:33:32.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.384+0000 7f8c6ae82640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8c640d1350 0x7f8c640d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.384+0000 7f8c6ae82640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8c640b7c90 0x7f8c640b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.384+0000 7f8c6ae82640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c6415e8b0 con 0x7f8c6415e4d0 2024-09-12T01:33:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.384+0000 7f8c6ae82640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f8c640d3bf0 con 0x7f8c640b7c90 2024-09-12T01:33:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.384+0000 7f8c6ae82640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f8c640d1d00 con 0x7f8c640d1350 2024-09-12T01:33:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.384+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8c640d1350 0x7f8c640d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.384+0000 7f8c6941f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8c640b7c90 0x7f8c640b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.384+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8c640d1350 0x7f8c640d17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44216/0 (socket says 172.21.15.16:44216) 2024-09-12T01:33:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.384+0000 7f8c68c1e640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8c6415e4d0 0x7f8c640cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:33426/0 (socket says 172.21.15.16:33426) 2024-09-12T01:33:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.384+0000 7f8c6941f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8c640b7c90 0x7f8c640b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45318/0 (socket says 172.21.15.16:45318) 2024-09-12T01:33:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.384+0000 7f8c63fff640 1 -- 172.21.15.16:0/1969186789 learned_addr learned my addr 172.21.15.16:0/1969186789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.385+0000 7f8c6941f640 1 -- 172.21.15.16:0/1969186789 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8c640d1350 msgr2=0x7f8c640d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.385+0000 7f8c6941f640 1 --2- 172.21.15.16:0/1969186789 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8c640d1350 0x7f8c640d17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.385+0000 7f8c6941f640 1 -- 172.21.15.16:0/1969186789 >> v1:172.21.15.33:6789/0 conn(0x7f8c6415e4d0 legacy=0x7f8c640cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.385+0000 7f8c6941f640 1 -- 172.21.15.16:0/1969186789 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c640b7510 con 0x7f8c640b7c90 2024-09-12T01:33:32.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.385+0000 7f8c637fe640 1 -- 172.21.15.16:0/1969186789 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 966256771 0 0) 0x7f8c6415e8b0 con 0x7f8c6415e4d0 2024-09-12T01:33:32.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.385+0000 7f8c63fff640 1 --2- 172.21.15.16:0/1969186789 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8c640d1350 0x7f8c640d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:32.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.385+0000 7f8c6941f640 1 --2- 172.21.15.16:0/1969186789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640b7c90 0x7f8c640b80c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c6415ea80 tx=0x7f8c5c02a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67738ce608821ed8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:32.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.386+0000 7f8c637fe640 1 -- 172.21.15.16:0/1969186789 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c02ae60 con 0x7f8c640b7c90 2024-09-12T01:33:32.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.386+0000 7f8c637fe640 1 -- 172.21.15.16:0/1969186789 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c5c007740 con 0x7f8c640b7c90 2024-09-12T01:33:32.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.386+0000 7f8c637fe640 1 -- 172.21.15.16:0/1969186789 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c007b10 con 0x7f8c640b7c90 2024-09-12T01:33:32.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.386+0000 7f8c6ae82640 1 -- 172.21.15.16:0/1969186789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640b7c90 msgr2=0x7f8c640b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.386+0000 7f8c6ae82640 1 --2- 172.21.15.16:0/1969186789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640b7c90 0x7f8c640b80c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c6415ea80 tx=0x7f8c5c02a640 comp rx=0 tx=0).stop 2024-09-12T01:33:32.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.386+0000 7f8c6ae82640 1 -- 172.21.15.16:0/1969186789 shutdown_connections 2024-09-12T01:33:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.386+0000 7f8c6ae82640 1 --2- 172.21.15.16:0/1969186789 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8c640d1350 0x7f8c640d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.386+0000 7f8c6ae82640 1 --2- 172.21.15.16:0/1969186789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640b7c90 0x7f8c640b80c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.386+0000 7f8c6ae82640 1 -- 172.21.15.16:0/1969186789 >> 172.21.15.16:0/1969186789 conn(0x7f8c64073640 msgr2=0x7f8c64073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.386+0000 7f8c6ae82640 1 -- 172.21.15.16:0/1969186789 shutdown_connections 2024-09-12T01:33:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.386+0000 7f8c6ae82640 1 -- 172.21.15.16:0/1969186789 wait complete. 2024-09-12T01:33:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.387+0000 7f8c6ae82640 1 Processor -- start 2024-09-12T01:33:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.387+0000 7f8c6ae82640 1 -- start start 2024-09-12T01:33:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.388+0000 7f8c6ae82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c640b7c90 0x7f8c6415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.388+0000 7f8c6ae82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640d1350 0x7f8c6414f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.388+0000 7f8c6ae82640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8c6415e4d0 0x7f8c640b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.388+0000 7f8c6ae82640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c640b77b0 con 0x7f8c640b7c90 2024-09-12T01:33:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.388+0000 7f8c6ae82640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c640d3100 con 0x7f8c640d1350 2024-09-12T01:33:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.388+0000 7f8c6ae82640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f8c640d3350 con 0x7f8c6415e4d0 2024-09-12T01:33:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c68c1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c640b7c90 0x7f8c6415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640d1350 0x7f8c6414f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c68c1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c640b7c90 0x7f8c6415bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42046/0 (socket says 172.21.15.16:42046) 2024-09-12T01:33:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c6941f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8c6415e4d0 0x7f8c640b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c68c1e640 1 -- 172.21.15.16:0/3502365398 learned_addr learned my addr 172.21.15.16:0/3502365398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c68c1e640 1 -- 172.21.15.16:0/3502365398 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8c6415e4d0 msgr2=0x7f8c640b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c68c1e640 1 --2- 172.21.15.16:0/3502365398 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8c6415e4d0 0x7f8c640b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c68c1e640 1 -- 172.21.15.16:0/3502365398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640d1350 msgr2=0x7f8c6414f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c68c1e640 1 --2- 172.21.15.16:0/3502365398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640d1350 0x7f8c6414f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c68c1e640 1 -- 172.21.15.16:0/3502365398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c641f76c0 con 0x7f8c640b7c90 2024-09-12T01:33:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c63fff640 1 --2- 172.21.15.16:0/3502365398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640d1350 0x7f8c6414f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c68c1e640 1 --2- 172.21.15.16:0/3502365398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c640b7c90 0x7f8c6415bff0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400c430 tx=0x7f8c5400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c617fa640 1 -- 172.21.15.16:0/3502365398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c540074e0 con 0x7f8c640b7c90 2024-09-12T01:33:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c617fa640 1 -- 172.21.15.16:0/3502365398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c540047e0 con 0x7f8c640b7c90 2024-09-12T01:33:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c617fa640 1 -- 172.21.15.16:0/3502365398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54004bd0 con 0x7f8c640b7c90 2024-09-12T01:33:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.389+0000 7f8c6ae82640 1 -- 172.21.15.16:0/3502365398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c641f79a0 con 0x7f8c640b7c90 2024-09-12T01:33:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.390+0000 7f8c6ae82640 1 -- 172.21.15.16:0/3502365398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c641f7f60 con 0x7f8c640b7c90 2024-09-12T01:33:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.391+0000 7f8c6ae82640 1 -- 172.21.15.16:0/3502365398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c641f8340 con 0x7f8c640b7c90 2024-09-12T01:33:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.391+0000 7f8c617fa640 1 -- 172.21.15.16:0/3502365398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c54011070 con 0x7f8c640b7c90 2024-09-12T01:33:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.391+0000 7f8c617fa640 1 --2- 172.21.15.16:0/3502365398 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8c5006be30 0x7f8c5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.392+0000 7f8c617fa640 1 -- 172.21.15.16:0/3502365398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8c5408acc0 con 0x7f8c640b7c90 2024-09-12T01:33:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.392+0000 7f8c63fff640 1 --2- 172.21.15.16:0/3502365398 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8c5006be30 0x7f8c5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.392+0000 7f8c63fff640 1 --2- 172.21.15.16:0/3502365398 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8c5006be30 0x7f8c5006e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009630 tx=0x7f8c58009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:32.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.396+0000 7f8c617fa640 1 -- 172.21.15.16:0/3502365398 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8c641f8340 con 0x7f8c640b7c90 2024-09-12T01:33:32.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.571+0000 7f8c6ae82640 1 -- 172.21.15.16:0/3502365398 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c641f7470 con 0x7f8c5006be30 2024-09-12T01:33:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.579+0000 7f8c617fa640 1 -- 172.21.15.16:0/3502365398 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f8c641f7470 con 0x7f8c5006be30 2024-09-12T01:33:32.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:33:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.582+0000 7f8c6ae82640 1 -- 172.21.15.16:0/3502365398 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8c5006be30 msgr2=0x7f8c5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.582+0000 7f8c6ae82640 1 --2- 172.21.15.16:0/3502365398 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8c5006be30 0x7f8c5006e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009630 tx=0x7f8c58009240 comp rx=0 tx=0).stop 2024-09-12T01:33:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.582+0000 7f8c6ae82640 1 -- 172.21.15.16:0/3502365398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c640b7c90 msgr2=0x7f8c6415bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.582+0000 7f8c6ae82640 1 --2- 172.21.15.16:0/3502365398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c640b7c90 0x7f8c6415bff0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400c430 tx=0x7f8c5400c8f0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.583+0000 7f8c6ae82640 1 -- 172.21.15.16:0/3502365398 shutdown_connections 2024-09-12T01:33:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.583+0000 7f8c6ae82640 1 --2- 172.21.15.16:0/3502365398 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8c5006be30 0x7f8c5006e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.583+0000 7f8c6ae82640 1 --2- 172.21.15.16:0/3502365398 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8c6415e4d0 0x7f8c640b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.583+0000 7f8c6ae82640 1 --2- 172.21.15.16:0/3502365398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640d1350 0x7f8c6414f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.583+0000 7f8c6ae82640 1 --2- 172.21.15.16:0/3502365398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c640b7c90 0x7f8c6415bff0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.583+0000 7f8c6ae82640 1 -- 172.21.15.16:0/3502365398 >> 172.21.15.16:0/3502365398 conn(0x7f8c64073640 msgr2=0x7f8c640b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.583+0000 7f8c6ae82640 1 -- 172.21.15.16:0/3502365398 shutdown_connections 2024-09-12T01:33:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.583+0000 7f8c6ae82640 1 -- 172.21.15.16:0/3502365398 wait complete. 2024-09-12T01:33:32.594 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-12T01:33:32.474774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236204,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738516,"statfs":{"total":773094113280,"available":772852240384,"internally_reserved":0,"allocated":3416064,"data_stored":2069540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002980"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413650+0000","last_change":"2024-09-12T01:33:11.148986+0000","last_active":"2024-09-12T01:33:19.413650+0000","last_peered":"2024-09-12T01:33:19.413650+0000","last_clean":"2024-09-12T01:33:19.413650+0000","last_became_active":"2024-09-12T01:33:11.146323+0000","last_became_peered":"2024-09-12T01:33:11.146323+0000","last_unstale":"2024-09-12T01:33:19.413650+0000","last_undegraded":"2024-09-12T01:33:19.413650+0000","last_fullsized":"2024-09-12T01:33:19.413650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:54.118009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217034+0000","last_change":"2024-09-12T01:33:12.183084+0000","last_active":"2024-09-12T01:33:18.217034+0000","last_peered":"2024-09-12T01:33:18.217034+0000","last_clean":"2024-09-12T01:33:18.217034+0000","last_became_active":"2024-09-12T01:33:12.182286+0000","last_became_peered":"2024-09-12T01:33:12.182286+0000","last_unstale":"2024-09-12T01:33:18.217034+0000","last_undegraded":"2024-09-12T01:33:18.217034+0000","last_fullsized":"2024-09-12T01:33:18.217034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:46.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216567+0000","last_change":"2024-09-12T01:33:16.593060+0000","last_active":"2024-09-12T01:33:18.216567+0000","last_peered":"2024-09-12T01:33:18.216567+0000","last_clean":"2024-09-12T01:33:18.216567+0000","last_became_active":"2024-09-12T01:33:16.592584+0000","last_became_peered":"2024-09-12T01:33:16.592584+0000","last_unstale":"2024-09-12T01:33:18.216567+0000","last_undegraded":"2024-09-12T01:33:18.216567+0000","last_fullsized":"2024-09-12T01:33:18.216567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:57.525309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217133+0000","last_change":"2024-09-12T01:33:11.142022+0000","last_active":"2024-09-12T01:33:18.217133+0000","last_peered":"2024-09-12T01:33:18.217133+0000","last_clean":"2024-09-12T01:33:18.217133+0000","last_became_active":"2024-09-12T01:33:11.141709+0000","last_became_peered":"2024-09-12T01:33:11.141709+0000","last_unstale":"2024-09-12T01:33:18.217133+0000","last_undegraded":"2024-09-12T01:33:18.217133+0000","last_fullsized":"2024-09-12T01:33:18.217133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:04.872785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413419+0000","last_change":"2024-09-12T01:33:12.176163+0000","last_active":"2024-09-12T01:33:19.413419+0000","last_peered":"2024-09-12T01:33:19.413419+0000","last_clean":"2024-09-12T01:33:19.413419+0000","last_became_active":"2024-09-12T01:33:12.175795+0000","last_became_peered":"2024-09-12T01:33:12.175795+0000","last_unstale":"2024-09-12T01:33:19.413419+0000","last_undegraded":"2024-09-12T01:33:19.413419+0000","last_fullsized":"2024-09-12T01:33:19.413419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:29.557033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217872+0000","last_change":"2024-09-12T01:33:16.202792+0000","last_active":"2024-09-12T01:33:18.217872+0000","last_peered":"2024-09-12T01:33:18.217872+0000","last_clean":"2024-09-12T01:33:18.217872+0000","last_became_active":"2024-09-12T01:33:16.202352+0000","last_became_peered":"2024-09-12T01:33:16.202352+0000","last_unstale":"2024-09-12T01:33:18.217872+0000","last_undegraded":"2024-09-12T01:33:18.217872+0000","last_fullsized":"2024-09-12T01:33:18.217872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:39.155565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487520+0000","last_change":"2024-09-12T01:33:11.149265+0000","last_active":"2024-09-12T01:33:19.487520+0000","last_peered":"2024-09-12T01:33:19.487520+0000","last_clean":"2024-09-12T01:33:19.487520+0000","last_became_active":"2024-09-12T01:33:11.148975+0000","last_became_peered":"2024-09-12T01:33:11.148975+0000","last_unstale":"2024-09-12T01:33:19.487520+0000","last_undegraded":"2024-09-12T01:33:19.487520+0000","last_fullsized":"2024-09-12T01:33:19.487520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:49.680053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839630+0000","last_change":"2024-09-12T01:33:12.175597+0000","last_active":"2024-09-12T01:33:18.839630+0000","last_peered":"2024-09-12T01:33:18.839630+0000","last_clean":"2024-09-12T01:33:18.839630+0000","last_became_active":"2024-09-12T01:33:12.175271+0000","last_became_peered":"2024-09-12T01:33:12.175271+0000","last_unstale":"2024-09-12T01:33:18.839630+0000","last_undegraded":"2024-09-12T01:33:18.839630+0000","last_fullsized":"2024-09-12T01:33:18.839630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:24.363602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218697+0000","last_change":"2024-09-12T01:33:16.200812+0000","last_active":"2024-09-12T01:33:18.218697+0000","last_peered":"2024-09-12T01:33:18.218697+0000","last_clean":"2024-09-12T01:33:18.218697+0000","last_became_active":"2024-09-12T01:33:16.200029+0000","last_became_peered":"2024-09-12T01:33:16.200029+0000","last_unstale":"2024-09-12T01:33:18.218697+0000","last_undegraded":"2024-09-12T01:33:18.218697+0000","last_fullsized":"2024-09-12T01:33:18.218697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:19.443781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485752+0000","last_change":"2024-09-12T01:33:11.153223+0000","last_active":"2024-09-12T01:33:19.485752+0000","last_peered":"2024-09-12T01:33:19.485752+0000","last_clean":"2024-09-12T01:33:19.485752+0000","last_became_active":"2024-09-12T01:33:11.152938+0000","last_became_peered":"2024-09-12T01:33:11.152938+0000","last_unstale":"2024-09-12T01:33:19.485752+0000","last_undegraded":"2024-09-12T01:33:19.485752+0000","last_fullsized":"2024-09-12T01:33:19.485752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839743+0000","last_change":"2024-09-12T01:33:12.178087+0000","last_active":"2024-09-12T01:33:18.839743+0000","last_peered":"2024-09-12T01:33:18.839743+0000","last_clean":"2024-09-12T01:33:18.839743+0000","last_became_active":"2024-09-12T01:33:12.176556+0000","last_became_peered":"2024-09-12T01:33:12.176556+0000","last_unstale":"2024-09-12T01:33:18.839743+0000","last_undegraded":"2024-09-12T01:33:18.839743+0000","last_fullsized":"2024-09-12T01:33:18.839743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:48.292206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218149+0000","last_change":"2024-09-12T01:33:16.193585+0000","last_active":"2024-09-12T01:33:18.218149+0000","last_peered":"2024-09-12T01:33:18.218149+0000","last_clean":"2024-09-12T01:33:18.218149+0000","last_became_active":"2024-09-12T01:33:16.193050+0000","last_became_peered":"2024-09-12T01:33:16.193050+0000","last_unstale":"2024-09-12T01:33:18.218149+0000","last_undegraded":"2024-09-12T01:33:18.218149+0000","last_fullsized":"2024-09-12T01:33:18.218149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:40.563121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486654+0000","last_change":"2024-09-12T01:33:12.178078+0000","last_active":"2024-09-12T01:33:19.486654+0000","last_peered":"2024-09-12T01:33:19.486654+0000","last_clean":"2024-09-12T01:33:19.486654+0000","last_became_active":"2024-09-12T01:33:12.177806+0000","last_became_peered":"2024-09-12T01:33:12.177806+0000","last_unstale":"2024-09-12T01:33:19.486654+0000","last_undegraded":"2024-09-12T01:33:19.486654+0000","last_fullsized":"2024-09-12T01:33:19.486654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:25.007983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485717+0000","last_change":"2024-09-12T01:33:11.149750+0000","last_active":"2024-09-12T01:33:19.485717+0000","last_peered":"2024-09-12T01:33:19.485717+0000","last_clean":"2024-09-12T01:33:19.485717+0000","last_became_active":"2024-09-12T01:33:11.149442+0000","last_became_peered":"2024-09-12T01:33:11.149442+0000","last_unstale":"2024-09-12T01:33:19.485717+0000","last_undegraded":"2024-09-12T01:33:19.485717+0000","last_fullsized":"2024-09-12T01:33:19.485717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218830+0000","last_change":"2024-09-12T01:33:16.202498+0000","last_active":"2024-09-12T01:33:18.218830+0000","last_peered":"2024-09-12T01:33:18.218830+0000","last_clean":"2024-09-12T01:33:18.218830+0000","last_became_active":"2024-09-12T01:33:16.200800+0000","last_became_peered":"2024-09-12T01:33:16.200800+0000","last_unstale":"2024-09-12T01:33:18.218830+0000","last_undegraded":"2024-09-12T01:33:18.218830+0000","last_fullsized":"2024-09-12T01:33:18.218830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:53.648825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216927+0000","last_change":"2024-09-12T01:33:12.169842+0000","last_active":"2024-09-12T01:33:18.216927+0000","last_peered":"2024-09-12T01:33:18.216927+0000","last_clean":"2024-09-12T01:33:18.216927+0000","last_became_active":"2024-09-12T01:33:12.169524+0000","last_became_peered":"2024-09-12T01:33:12.169524+0000","last_unstale":"2024-09-12T01:33:18.216927+0000","last_undegraded":"2024-09-12T01:33:18.216927+0000","last_fullsized":"2024-09-12T01:33:18.216927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:38.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485299+0000","last_change":"2024-09-12T01:33:11.147221+0000","last_active":"2024-09-12T01:33:19.485299+0000","last_peered":"2024-09-12T01:33:19.485299+0000","last_clean":"2024-09-12T01:33:19.485299+0000","last_became_active":"2024-09-12T01:33:11.146928+0000","last_became_peered":"2024-09-12T01:33:11.146928+0000","last_unstale":"2024-09-12T01:33:19.485299+0000","last_undegraded":"2024-09-12T01:33:19.485299+0000","last_fullsized":"2024-09-12T01:33:19.485299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:22.885518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218288+0000","last_change":"2024-09-12T01:33:16.196589+0000","last_active":"2024-09-12T01:33:18.218288+0000","last_peered":"2024-09-12T01:33:18.218288+0000","last_clean":"2024-09-12T01:33:18.218288+0000","last_became_active":"2024-09-12T01:33:16.196092+0000","last_became_peered":"2024-09-12T01:33:16.196092+0000","last_unstale":"2024-09-12T01:33:18.218288+0000","last_undegraded":"2024-09-12T01:33:18.218288+0000","last_fullsized":"2024-09-12T01:33:18.218288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:28.321407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216715+0000","last_change":"2024-09-12T01:33:12.165178+0000","last_active":"2024-09-12T01:33:18.216715+0000","last_peered":"2024-09-12T01:33:18.216715+0000","last_clean":"2024-09-12T01:33:18.216715+0000","last_became_active":"2024-09-12T01:33:12.162823+0000","last_became_peered":"2024-09-12T01:33:12.162823+0000","last_unstale":"2024-09-12T01:33:18.216715+0000","last_undegraded":"2024-09-12T01:33:18.216715+0000","last_fullsized":"2024-09-12T01:33:18.216715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:54.679349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485555+0000","last_change":"2024-09-12T01:33:11.145358+0000","last_active":"2024-09-12T01:33:19.485555+0000","last_peered":"2024-09-12T01:33:19.485555+0000","last_clean":"2024-09-12T01:33:19.485555+0000","last_became_active":"2024-09-12T01:33:11.145035+0000","last_became_peered":"2024-09-12T01:33:11.145035+0000","last_unstale":"2024-09-12T01:33:19.485555+0000","last_undegraded":"2024-09-12T01:33:19.485555+0000","last_fullsized":"2024-09-12T01:33:19.485555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486406+0000","last_change":"2024-09-12T01:33:16.594344+0000","last_active":"2024-09-12T01:33:19.486406+0000","last_peered":"2024-09-12T01:33:19.486406+0000","last_clean":"2024-09-12T01:33:19.486406+0000","last_became_active":"2024-09-12T01:33:16.593980+0000","last_became_peered":"2024-09-12T01:33:16.593980+0000","last_unstale":"2024-09-12T01:33:19.486406+0000","last_undegraded":"2024-09-12T01:33:19.486406+0000","last_fullsized":"2024-09-12T01:33:19.486406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:14.088579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218224+0000","last_change":"2024-09-12T01:33:12.170635+0000","last_active":"2024-09-12T01:33:18.218224+0000","last_peered":"2024-09-12T01:33:18.218224+0000","last_clean":"2024-09-12T01:33:18.218224+0000","last_became_active":"2024-09-12T01:33:12.169588+0000","last_became_peered":"2024-09-12T01:33:12.169588+0000","last_unstale":"2024-09-12T01:33:18.218224+0000","last_undegraded":"2024-09-12T01:33:18.218224+0000","last_fullsized":"2024-09-12T01:33:18.218224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:50.002525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486441+0000","last_change":"2024-09-12T01:33:11.151147+0000","last_active":"2024-09-12T01:33:19.486441+0000","last_peered":"2024-09-12T01:33:19.486441+0000","last_clean":"2024-09-12T01:33:19.486441+0000","last_became_active":"2024-09-12T01:33:11.150915+0000","last_became_peered":"2024-09-12T01:33:11.150915+0000","last_unstale":"2024-09-12T01:33:19.486441+0000","last_undegraded":"2024-09-12T01:33:19.486441+0000","last_fullsized":"2024-09-12T01:33:19.486441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:56.100575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218568+0000","last_change":"2024-09-12T01:33:16.202342+0000","last_active":"2024-09-12T01:33:18.218568+0000","last_peered":"2024-09-12T01:33:18.218568+0000","last_clean":"2024-09-12T01:33:18.218568+0000","last_became_active":"2024-09-12T01:33:16.200265+0000","last_became_peered":"2024-09-12T01:33:16.200265+0000","last_unstale":"2024-09-12T01:33:18.218568+0000","last_undegraded":"2024-09-12T01:33:18.218568+0000","last_fullsized":"2024-09-12T01:33:18.218568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:13.593524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413572+0000","last_change":"2024-09-12T01:33:12.171306+0000","last_active":"2024-09-12T01:33:19.413572+0000","last_peered":"2024-09-12T01:33:19.413572+0000","last_clean":"2024-09-12T01:33:19.413572+0000","last_became_active":"2024-09-12T01:33:12.171026+0000","last_became_peered":"2024-09-12T01:33:12.171026+0000","last_unstale":"2024-09-12T01:33:19.413572+0000","last_undegraded":"2024-09-12T01:33:19.413572+0000","last_fullsized":"2024-09-12T01:33:19.413572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:02.272439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487629+0000","last_change":"2024-09-12T01:33:11.153620+0000","last_active":"2024-09-12T01:33:19.487629+0000","last_peered":"2024-09-12T01:33:19.487629+0000","last_clean":"2024-09-12T01:33:19.487629+0000","last_became_active":"2024-09-12T01:33:11.153278+0000","last_became_peered":"2024-09-12T01:33:11.153278+0000","last_unstale":"2024-09-12T01:33:19.487629+0000","last_undegraded":"2024-09-12T01:33:19.487629+0000","last_fullsized":"2024-09-12T01:33:19.487629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217980+0000","last_change":"2024-09-12T01:33:16.590320+0000","last_active":"2024-09-12T01:33:18.217980+0000","last_peered":"2024-09-12T01:33:18.217980+0000","last_clean":"2024-09-12T01:33:18.217980+0000","last_became_active":"2024-09-12T01:33:16.589806+0000","last_became_peered":"2024-09-12T01:33:16.589806+0000","last_unstale":"2024-09-12T01:33:18.217980+0000","last_undegraded":"2024-09-12T01:33:18.217980+0000","last_fullsized":"2024-09-12T01:33:18.217980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:01.750852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217345+0000","last_change":"2024-09-12T01:33:12.177022+0000","last_active":"2024-09-12T01:33:18.217345+0000","last_peered":"2024-09-12T01:33:18.217345+0000","last_clean":"2024-09-12T01:33:18.217345+0000","last_became_active":"2024-09-12T01:33:12.176544+0000","last_became_peered":"2024-09-12T01:33:12.176544+0000","last_unstale":"2024-09-12T01:33:18.217345+0000","last_undegraded":"2024-09-12T01:33:18.217345+0000","last_fullsized":"2024-09-12T01:33:18.217345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:51.254891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486161+0000","last_change":"2024-09-12T01:33:11.144439+0000","last_active":"2024-09-12T01:33:19.486161+0000","last_peered":"2024-09-12T01:33:19.486161+0000","last_clean":"2024-09-12T01:33:19.486161+0000","last_became_active":"2024-09-12T01:33:11.144134+0000","last_became_peered":"2024-09-12T01:33:11.144134+0000","last_unstale":"2024-09-12T01:33:19.486161+0000","last_undegraded":"2024-09-12T01:33:19.486161+0000","last_fullsized":"2024-09-12T01:33:19.486161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216491+0000","last_change":"2024-09-12T01:33:16.588984+0000","last_active":"2024-09-12T01:33:18.216491+0000","last_peered":"2024-09-12T01:33:18.216491+0000","last_clean":"2024-09-12T01:33:18.216491+0000","last_became_active":"2024-09-12T01:33:16.588531+0000","last_became_peered":"2024-09-12T01:33:16.588531+0000","last_unstale":"2024-09-12T01:33:18.216491+0000","last_undegraded":"2024-09-12T01:33:18.216491+0000","last_fullsized":"2024-09-12T01:33:18.216491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:35.903448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486153+0000","last_change":"2024-09-12T01:33:12.162118+0000","last_active":"2024-09-12T01:33:19.486153+0000","last_peered":"2024-09-12T01:33:19.486153+0000","last_clean":"2024-09-12T01:33:19.486153+0000","last_became_active":"2024-09-12T01:33:12.161817+0000","last_became_peered":"2024-09-12T01:33:12.161817+0000","last_unstale":"2024-09-12T01:33:19.486153+0000","last_undegraded":"2024-09-12T01:33:19.486153+0000","last_fullsized":"2024-09-12T01:33:19.486153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:54.679349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487612+0000","last_change":"2024-09-12T01:33:11.150724+0000","last_active":"2024-09-12T01:33:19.487612+0000","last_peered":"2024-09-12T01:33:19.487612+0000","last_clean":"2024-09-12T01:33:19.487612+0000","last_became_active":"2024-09-12T01:33:11.150444+0000","last_became_peered":"2024-09-12T01:33:11.150444+0000","last_unstale":"2024-09-12T01:33:19.487612+0000","last_undegraded":"2024-09-12T01:33:19.487612+0000","last_fullsized":"2024-09-12T01:33:19.487612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:26.579494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413397+0000","last_change":"2024-09-12T01:33:16.589570+0000","last_active":"2024-09-12T01:33:19.413397+0000","last_peered":"2024-09-12T01:33:19.413397+0000","last_clean":"2024-09-12T01:33:19.413397+0000","last_became_active":"2024-09-12T01:33:16.589137+0000","last_became_peered":"2024-09-12T01:33:16.589137+0000","last_unstale":"2024-09-12T01:33:19.413397+0000","last_undegraded":"2024-09-12T01:33:19.413397+0000","last_fullsized":"2024-09-12T01:33:19.413397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:31:18.804222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839884+0000","last_change":"2024-09-12T01:33:12.174625+0000","last_active":"2024-09-12T01:33:18.839884+0000","last_peered":"2024-09-12T01:33:18.839884+0000","last_clean":"2024-09-12T01:33:18.839884+0000","last_became_active":"2024-09-12T01:33:12.174308+0000","last_became_peered":"2024-09-12T01:33:12.174308+0000","last_unstale":"2024-09-12T01:33:18.839884+0000","last_undegraded":"2024-09-12T01:33:18.839884+0000","last_fullsized":"2024-09-12T01:33:18.839884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:24.134321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486236+0000","last_change":"2024-09-12T01:33:11.142280+0000","last_active":"2024-09-12T01:33:19.486236+0000","last_peered":"2024-09-12T01:33:19.486236+0000","last_clean":"2024-09-12T01:33:19.486236+0000","last_became_active":"2024-09-12T01:33:11.141976+0000","last_became_peered":"2024-09-12T01:33:11.141976+0000","last_unstale":"2024-09-12T01:33:19.486236+0000","last_undegraded":"2024-09-12T01:33:19.486236+0000","last_fullsized":"2024-09-12T01:33:19.486236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:11.527363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485800+0000","last_change":"2024-09-12T01:33:16.200223+0000","last_active":"2024-09-12T01:33:19.485800+0000","last_peered":"2024-09-12T01:33:19.485800+0000","last_clean":"2024-09-12T01:33:19.485800+0000","last_became_active":"2024-09-12T01:33:16.199767+0000","last_became_peered":"2024-09-12T01:33:16.199767+0000","last_unstale":"2024-09-12T01:33:19.485800+0000","last_undegraded":"2024-09-12T01:33:19.485800+0000","last_fullsized":"2024-09-12T01:33:19.485800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:01.750852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218829+0000","last_change":"2024-09-12T01:33:12.171774+0000","last_active":"2024-09-12T01:33:18.218829+0000","last_peered":"2024-09-12T01:33:18.218829+0000","last_clean":"2024-09-12T01:33:18.218829+0000","last_became_active":"2024-09-12T01:33:12.171472+0000","last_became_peered":"2024-09-12T01:33:12.171472+0000","last_unstale":"2024-09-12T01:33:18.218829+0000","last_undegraded":"2024-09-12T01:33:18.218829+0000","last_fullsized":"2024-09-12T01:33:18.218829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:50.682111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839456+0000","last_change":"2024-09-12T01:33:11.146150+0000","last_active":"2024-09-12T01:33:18.839456+0000","last_peered":"2024-09-12T01:33:18.839456+0000","last_clean":"2024-09-12T01:33:18.839456+0000","last_became_active":"2024-09-12T01:33:11.145690+0000","last_became_peered":"2024-09-12T01:33:11.145690+0000","last_unstale":"2024-09-12T01:33:18.839456+0000","last_undegraded":"2024-09-12T01:33:18.839456+0000","last_fullsized":"2024-09-12T01:33:18.839456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:49.680053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486103+0000","last_change":"2024-09-12T01:33:16.200773+0000","last_active":"2024-09-12T01:33:19.486103+0000","last_peered":"2024-09-12T01:33:19.486103+0000","last_clean":"2024-09-12T01:33:19.486103+0000","last_became_active":"2024-09-12T01:33:16.199757+0000","last_became_peered":"2024-09-12T01:33:16.199757+0000","last_unstale":"2024-09-12T01:33:19.486103+0000","last_undegraded":"2024-09-12T01:33:19.486103+0000","last_fullsized":"2024-09-12T01:33:19.486103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:40.563121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485669+0000","last_change":"2024-09-12T01:33:12.162181+0000","last_active":"2024-09-12T01:33:19.485669+0000","last_peered":"2024-09-12T01:33:19.485669+0000","last_clean":"2024-09-12T01:33:19.485669+0000","last_became_active":"2024-09-12T01:33:12.161900+0000","last_became_peered":"2024-09-12T01:33:12.161900+0000","last_unstale":"2024-09-12T01:33:19.485669+0000","last_undegraded":"2024-09-12T01:33:19.485669+0000","last_fullsized":"2024-09-12T01:33:19.485669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:52.439908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554120+0000","last_change":"2024-09-12T01:33:11.147499+0000","last_active":"2024-09-12T01:33:18.554120+0000","last_peered":"2024-09-12T01:33:18.554120+0000","last_clean":"2024-09-12T01:33:18.554120+0000","last_became_active":"2024-09-12T01:33:11.147186+0000","last_became_peered":"2024-09-12T01:33:11.147186+0000","last_unstale":"2024-09-12T01:33:18.554120+0000","last_undegraded":"2024-09-12T01:33:18.554120+0000","last_fullsized":"2024-09-12T01:33:18.554120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:15.921424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486454+0000","last_change":"2024-09-12T01:33:16.201262+0000","last_active":"2024-09-12T01:33:19.486454+0000","last_peered":"2024-09-12T01:33:19.486454+0000","last_clean":"2024-09-12T01:33:19.486454+0000","last_became_active":"2024-09-12T01:33:16.200850+0000","last_became_peered":"2024-09-12T01:33:16.200850+0000","last_unstale":"2024-09-12T01:33:19.486454+0000","last_undegraded":"2024-09-12T01:33:19.486454+0000","last_fullsized":"2024-09-12T01:33:19.486454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:13.593524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.412948+0000","last_change":"2024-09-12T01:33:12.177601+0000","last_active":"2024-09-12T01:33:19.412948+0000","last_peered":"2024-09-12T01:33:19.412948+0000","last_clean":"2024-09-12T01:33:19.412948+0000","last_became_active":"2024-09-12T01:33:12.177310+0000","last_became_peered":"2024-09-12T01:33:12.177310+0000","last_unstale":"2024-09-12T01:33:19.412948+0000","last_undegraded":"2024-09-12T01:33:19.412948+0000","last_fullsized":"2024-09-12T01:33:19.412948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:59.541178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486689+0000","last_change":"2024-09-12T01:33:11.151233+0000","last_active":"2024-09-12T01:33:19.486689+0000","last_peered":"2024-09-12T01:33:19.486689+0000","last_clean":"2024-09-12T01:33:19.486689+0000","last_became_active":"2024-09-12T01:33:11.150960+0000","last_became_peered":"2024-09-12T01:33:11.150960+0000","last_unstale":"2024-09-12T01:33:19.486689+0000","last_undegraded":"2024-09-12T01:33:19.486689+0000","last_fullsized":"2024-09-12T01:33:19.486689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413558+0000","last_change":"2024-09-12T01:33:16.593340+0000","last_active":"2024-09-12T01:33:19.413558+0000","last_peered":"2024-09-12T01:33:19.413558+0000","last_clean":"2024-09-12T01:33:19.413558+0000","last_became_active":"2024-09-12T01:33:16.592942+0000","last_became_peered":"2024-09-12T01:33:16.592942+0000","last_unstale":"2024-09-12T01:33:19.413558+0000","last_undegraded":"2024-09-12T01:33:19.413558+0000","last_fullsized":"2024-09-12T01:33:19.413558+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:06.180609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216813+0000","last_change":"2024-09-12T01:33:12.175958+0000","last_active":"2024-09-12T01:33:18.216813+0000","last_peered":"2024-09-12T01:33:18.216813+0000","last_clean":"2024-09-12T01:33:18.216813+0000","last_became_active":"2024-09-12T01:33:12.174098+0000","last_became_peered":"2024-09-12T01:33:12.174098+0000","last_unstale":"2024-09-12T01:33:18.216813+0000","last_undegraded":"2024-09-12T01:33:18.216813+0000","last_fullsized":"2024-09-12T01:33:18.216813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839416+0000","last_change":"2024-09-12T01:33:11.148301+0000","last_active":"2024-09-12T01:33:18.839416+0000","last_peered":"2024-09-12T01:33:18.839416+0000","last_clean":"2024-09-12T01:33:18.839416+0000","last_became_active":"2024-09-12T01:33:11.146483+0000","last_became_peered":"2024-09-12T01:33:11.146483+0000","last_unstale":"2024-09-12T01:33:18.839416+0000","last_undegraded":"2024-09-12T01:33:18.839416+0000","last_fullsized":"2024-09-12T01:33:18.839416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:11.527363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486566+0000","last_change":"2024-09-12T01:33:01.096542+0000","last_active":"2024-09-12T01:33:19.486566+0000","last_peered":"2024-09-12T01:33:19.486566+0000","last_clean":"2024-09-12T01:33:19.486566+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:33:19.486566+0000","last_undegraded":"2024-09-12T01:33:19.486566+0000","last_fullsized":"2024-09-12T01:33:19.486566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_clean_scrub_stamp":"2024-09-12T01:32:58.512553+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:48:58.741813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486931+0000","last_change":"2024-09-12T01:33:16.591360+0000","last_active":"2024-09-12T01:33:19.486931+0000","last_peered":"2024-09-12T01:33:19.486931+0000","last_clean":"2024-09-12T01:33:19.486931+0000","last_became_active":"2024-09-12T01:33:16.590884+0000","last_became_peered":"2024-09-12T01:33:16.590884+0000","last_unstale":"2024-09-12T01:33:19.486931+0000","last_undegraded":"2024-09-12T01:33:19.486931+0000","last_fullsized":"2024-09-12T01:33:19.486931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:15.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487399+0000","last_change":"2024-09-12T01:33:12.178384+0000","last_active":"2024-09-12T01:33:19.487399+0000","last_peered":"2024-09-12T01:33:19.487399+0000","last_clean":"2024-09-12T01:33:19.487399+0000","last_became_active":"2024-09-12T01:33:12.178068+0000","last_became_peered":"2024-09-12T01:33:12.178068+0000","last_unstale":"2024-09-12T01:33:19.487399+0000","last_undegraded":"2024-09-12T01:33:19.487399+0000","last_fullsized":"2024-09-12T01:33:19.487399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:51.254891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554226+0000","last_change":"2024-09-12T01:33:11.149796+0000","last_active":"2024-09-12T01:33:18.554226+0000","last_peered":"2024-09-12T01:33:18.554226+0000","last_clean":"2024-09-12T01:33:18.554226+0000","last_became_active":"2024-09-12T01:33:11.149505+0000","last_became_peered":"2024-09-12T01:33:11.149505+0000","last_unstale":"2024-09-12T01:33:18.554226+0000","last_undegraded":"2024-09-12T01:33:18.554226+0000","last_fullsized":"2024-09-12T01:33:18.554226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:24.957500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486471+0000","last_change":"2024-09-12T01:33:16.589125+0000","last_active":"2024-09-12T01:33:19.486471+0000","last_peered":"2024-09-12T01:33:19.486471+0000","last_clean":"2024-09-12T01:33:19.486471+0000","last_became_active":"2024-09-12T01:33:16.588637+0000","last_became_peered":"2024-09-12T01:33:16.588637+0000","last_unstale":"2024-09-12T01:33:19.486471+0000","last_undegraded":"2024-09-12T01:33:19.486471+0000","last_fullsized":"2024-09-12T01:33:19.486471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:06.997893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839688+0000","last_change":"2024-09-12T01:33:12.176967+0000","last_active":"2024-09-12T01:33:18.839688+0000","last_peered":"2024-09-12T01:33:18.839688+0000","last_clean":"2024-09-12T01:33:18.839688+0000","last_became_active":"2024-09-12T01:33:12.176355+0000","last_became_peered":"2024-09-12T01:33:12.176355+0000","last_unstale":"2024-09-12T01:33:18.839688+0000","last_undegraded":"2024-09-12T01:33:18.839688+0000","last_fullsized":"2024-09-12T01:33:18.839688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:57.532127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485317+0000","last_change":"2024-09-12T01:33:11.149279+0000","last_active":"2024-09-12T01:33:19.485317+0000","last_peered":"2024-09-12T01:33:19.485317+0000","last_clean":"2024-09-12T01:33:19.485317+0000","last_became_active":"2024-09-12T01:33:11.148215+0000","last_became_peered":"2024-09-12T01:33:11.148215+0000","last_unstale":"2024-09-12T01:33:19.485317+0000","last_undegraded":"2024-09-12T01:33:19.485317+0000","last_fullsized":"2024-09-12T01:33:19.485317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:49.680053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413355+0000","last_change":"2024-09-12T01:33:16.200672+0000","last_active":"2024-09-12T01:33:19.413355+0000","last_peered":"2024-09-12T01:33:19.413355+0000","last_clean":"2024-09-12T01:33:19.413355+0000","last_became_active":"2024-09-12T01:33:16.200251+0000","last_became_peered":"2024-09-12T01:33:16.200251+0000","last_unstale":"2024-09-12T01:33:19.413355+0000","last_undegraded":"2024-09-12T01:33:19.413355+0000","last_fullsized":"2024-09-12T01:33:19.413355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:37.500171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487810+0000","last_change":"2024-09-12T01:33:12.179113+0000","last_active":"2024-09-12T01:33:19.487810+0000","last_peered":"2024-09-12T01:33:19.487810+0000","last_clean":"2024-09-12T01:33:19.487810+0000","last_became_active":"2024-09-12T01:33:12.178495+0000","last_became_peered":"2024-09-12T01:33:12.178495+0000","last_unstale":"2024-09-12T01:33:19.487810+0000","last_undegraded":"2024-09-12T01:33:19.487810+0000","last_fullsized":"2024-09-12T01:33:19.487810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:51:57.703184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554326+0000","last_change":"2024-09-12T01:33:11.144585+0000","last_active":"2024-09-12T01:33:18.554326+0000","last_peered":"2024-09-12T01:33:18.554326+0000","last_clean":"2024-09-12T01:33:18.554326+0000","last_became_active":"2024-09-12T01:33:11.144277+0000","last_became_peered":"2024-09-12T01:33:11.144277+0000","last_unstale":"2024-09-12T01:33:18.554326+0000","last_undegraded":"2024-09-12T01:33:18.554326+0000","last_fullsized":"2024-09-12T01:33:18.554326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:15.342457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839029+0000","last_change":"2024-09-12T01:33:16.591894+0000","last_active":"2024-09-12T01:33:18.839029+0000","last_peered":"2024-09-12T01:33:18.839029+0000","last_clean":"2024-09-12T01:33:18.839029+0000","last_became_active":"2024-09-12T01:33:16.591452+0000","last_became_peered":"2024-09-12T01:33:16.591452+0000","last_unstale":"2024-09-12T01:33:18.839029+0000","last_undegraded":"2024-09-12T01:33:18.839029+0000","last_fullsized":"2024-09-12T01:33:18.839029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:59.574294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487365+0000","last_change":"2024-09-12T01:33:12.176012+0000","last_active":"2024-09-12T01:33:19.487365+0000","last_peered":"2024-09-12T01:33:19.487365+0000","last_clean":"2024-09-12T01:33:19.487365+0000","last_became_active":"2024-09-12T01:33:12.175729+0000","last_became_peered":"2024-09-12T01:33:12.175729+0000","last_unstale":"2024-09-12T01:33:19.487365+0000","last_undegraded":"2024-09-12T01:33:19.487365+0000","last_fullsized":"2024-09-12T01:33:19.487365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:57.532127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.219006+0000","last_change":"2024-09-12T01:33:11.138837+0000","last_active":"2024-09-12T01:33:18.219006+0000","last_peered":"2024-09-12T01:33:18.219006+0000","last_clean":"2024-09-12T01:33:18.219006+0000","last_became_active":"2024-09-12T01:33:11.138324+0000","last_became_peered":"2024-09-12T01:33:11.138324+0000","last_unstale":"2024-09-12T01:33:18.219006+0000","last_undegraded":"2024-09-12T01:33:18.219006+0000","last_fullsized":"2024-09-12T01:33:18.219006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:48.706430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217732+0000","last_change":"2024-09-12T01:33:16.200428+0000","last_active":"2024-09-12T01:33:18.217732+0000","last_peered":"2024-09-12T01:33:18.217732+0000","last_clean":"2024-09-12T01:33:18.217732+0000","last_became_active":"2024-09-12T01:33:16.199943+0000","last_became_peered":"2024-09-12T01:33:16.199943+0000","last_unstale":"2024-09-12T01:33:18.217732+0000","last_undegraded":"2024-09-12T01:33:18.217732+0000","last_fullsized":"2024-09-12T01:33:18.217732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:15.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486512+0000","last_change":"2024-09-12T01:33:12.176546+0000","last_active":"2024-09-12T01:33:19.486512+0000","last_peered":"2024-09-12T01:33:19.486512+0000","last_clean":"2024-09-12T01:33:19.486512+0000","last_became_active":"2024-09-12T01:33:12.176254+0000","last_became_peered":"2024-09-12T01:33:12.176254+0000","last_unstale":"2024-09-12T01:33:19.486512+0000","last_undegraded":"2024-09-12T01:33:19.486512+0000","last_fullsized":"2024-09-12T01:33:19.486512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:24.363602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.553875+0000","last_change":"2024-09-12T01:33:11.143423+0000","last_active":"2024-09-12T01:33:18.553875+0000","last_peered":"2024-09-12T01:33:18.553875+0000","last_clean":"2024-09-12T01:33:18.553875+0000","last_became_active":"2024-09-12T01:33:11.143125+0000","last_became_peered":"2024-09-12T01:33:11.143125+0000","last_unstale":"2024-09-12T01:33:18.553875+0000","last_undegraded":"2024-09-12T01:33:18.553875+0000","last_fullsized":"2024-09-12T01:33:18.553875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:24.005795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413405+0000","last_change":"2024-09-12T01:33:16.190124+0000","last_active":"2024-09-12T01:33:19.413405+0000","last_peered":"2024-09-12T01:33:19.413405+0000","last_clean":"2024-09-12T01:33:19.413405+0000","last_became_active":"2024-09-12T01:33:16.189728+0000","last_became_peered":"2024-09-12T01:33:16.189728+0000","last_unstale":"2024-09-12T01:33:19.413405+0000","last_undegraded":"2024-09-12T01:33:19.413405+0000","last_fullsized":"2024-09-12T01:33:19.413405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:00.559557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486163+0000","last_change":"2024-09-12T01:33:12.173415+0000","last_active":"2024-09-12T01:33:19.486163+0000","last_peered":"2024-09-12T01:33:19.486163+0000","last_clean":"2024-09-12T01:33:19.486163+0000","last_became_active":"2024-09-12T01:33:12.173117+0000","last_became_peered":"2024-09-12T01:33:12.173117+0000","last_unstale":"2024-09-12T01:33:19.486163+0000","last_undegraded":"2024-09-12T01:33:19.486163+0000","last_fullsized":"2024-09-12T01:33:19.486163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:54.679349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486025+0000","last_change":"2024-09-12T01:33:11.140718+0000","last_active":"2024-09-12T01:33:19.486025+0000","last_peered":"2024-09-12T01:33:19.486025+0000","last_clean":"2024-09-12T01:33:19.486025+0000","last_became_active":"2024-09-12T01:33:11.140321+0000","last_became_peered":"2024-09-12T01:33:11.140321+0000","last_unstale":"2024-09-12T01:33:19.486025+0000","last_undegraded":"2024-09-12T01:33:19.486025+0000","last_fullsized":"2024-09-12T01:33:19.486025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:22.885518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217295+0000","last_change":"2024-09-12T01:33:16.594875+0000","last_active":"2024-09-12T01:33:18.217295+0000","last_peered":"2024-09-12T01:33:18.217295+0000","last_clean":"2024-09-12T01:33:18.217295+0000","last_became_active":"2024-09-12T01:33:16.592733+0000","last_became_peered":"2024-09-12T01:33:16.592733+0000","last_unstale":"2024-09-12T01:33:18.217295+0000","last_undegraded":"2024-09-12T01:33:18.217295+0000","last_fullsized":"2024-09-12T01:33:18.217295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:39.080379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486853+0000","last_change":"2024-09-12T01:33:12.175983+0000","last_active":"2024-09-12T01:33:19.486853+0000","last_peered":"2024-09-12T01:33:19.486853+0000","last_clean":"2024-09-12T01:33:19.486853+0000","last_became_active":"2024-09-12T01:33:12.175638+0000","last_became_peered":"2024-09-12T01:33:12.175638+0000","last_unstale":"2024-09-12T01:33:19.486853+0000","last_undegraded":"2024-09-12T01:33:19.486853+0000","last_fullsized":"2024-09-12T01:33:19.486853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:48.292206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839432+0000","last_change":"2024-09-12T01:33:11.148832+0000","last_active":"2024-09-12T01:33:18.839432+0000","last_peered":"2024-09-12T01:33:18.839432+0000","last_clean":"2024-09-12T01:33:18.839432+0000","last_became_active":"2024-09-12T01:33:11.148519+0000","last_became_peered":"2024-09-12T01:33:11.148519+0000","last_unstale":"2024-09-12T01:33:18.839432+0000","last_undegraded":"2024-09-12T01:33:18.839432+0000","last_fullsized":"2024-09-12T01:33:18.839432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217452+0000","last_change":"2024-09-12T01:33:16.588958+0000","last_active":"2024-09-12T01:33:18.217452+0000","last_peered":"2024-09-12T01:33:18.217452+0000","last_clean":"2024-09-12T01:33:18.217452+0000","last_became_active":"2024-09-12T01:33:16.588510+0000","last_became_peered":"2024-09-12T01:33:16.588510+0000","last_unstale":"2024-09-12T01:33:18.217452+0000","last_undegraded":"2024-09-12T01:33:18.217452+0000","last_fullsized":"2024-09-12T01:33:18.217452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:00.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413221+0000","last_change":"2024-09-12T01:33:12.184393+0000","last_active":"2024-09-12T01:33:19.413221+0000","last_peered":"2024-09-12T01:33:19.413221+0000","last_clean":"2024-09-12T01:33:19.413221+0000","last_became_active":"2024-09-12T01:33:12.184103+0000","last_became_peered":"2024-09-12T01:33:12.184103+0000","last_unstale":"2024-09-12T01:33:19.413221+0000","last_undegraded":"2024-09-12T01:33:19.413221+0000","last_fullsized":"2024-09-12T01:33:19.413221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:36.026056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.412683+0000","last_change":"2024-09-12T01:33:11.134020+0000","last_active":"2024-09-12T01:33:19.412683+0000","last_peered":"2024-09-12T01:33:19.412683+0000","last_clean":"2024-09-12T01:33:19.412683+0000","last_became_active":"2024-09-12T01:33:11.133717+0000","last_became_peered":"2024-09-12T01:33:11.133717+0000","last_unstale":"2024-09-12T01:33:19.412683+0000","last_undegraded":"2024-09-12T01:33:19.412683+0000","last_fullsized":"2024-09-12T01:33:19.412683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:20.901290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839735+0000","last_change":"2024-09-12T01:33:16.593051+0000","last_active":"2024-09-12T01:33:18.839735+0000","last_peered":"2024-09-12T01:33:18.839735+0000","last_clean":"2024-09-12T01:33:18.839735+0000","last_became_active":"2024-09-12T01:33:16.592513+0000","last_became_peered":"2024-09-12T01:33:16.592513+0000","last_unstale":"2024-09-12T01:33:18.839735+0000","last_undegraded":"2024-09-12T01:33:18.839735+0000","last_fullsized":"2024-09-12T01:33:18.839735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:35.903448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487266+0000","last_change":"2024-09-12T01:33:12.178596+0000","last_active":"2024-09-12T01:33:19.487266+0000","last_peered":"2024-09-12T01:33:19.487266+0000","last_clean":"2024-09-12T01:33:19.487266+0000","last_became_active":"2024-09-12T01:33:12.176316+0000","last_became_peered":"2024-09-12T01:33:12.176316+0000","last_unstale":"2024-09-12T01:33:19.487266+0000","last_undegraded":"2024-09-12T01:33:19.487266+0000","last_fullsized":"2024-09-12T01:33:19.487266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:24.134321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:21.709458+0000","last_change":"2024-09-12T01:33:11.147275+0000","last_active":"2024-09-12T01:33:21.709458+0000","last_peered":"2024-09-12T01:33:21.709458+0000","last_clean":"2024-09-12T01:33:21.709458+0000","last_became_active":"2024-09-12T01:33:11.146962+0000","last_became_peered":"2024-09-12T01:33:11.146962+0000","last_unstale":"2024-09-12T01:33:21.709458+0000","last_undegraded":"2024-09-12T01:33:21.709458+0000","last_fullsized":"2024-09-12T01:33:21.709458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:11.527363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485898+0000","last_change":"2024-09-12T01:33:16.202736+0000","last_active":"2024-09-12T01:33:19.485898+0000","last_peered":"2024-09-12T01:33:19.485898+0000","last_clean":"2024-09-12T01:33:19.485898+0000","last_became_active":"2024-09-12T01:33:16.202282+0000","last_became_peered":"2024-09-12T01:33:16.202282+0000","last_unstale":"2024-09-12T01:33:19.485898+0000","last_undegraded":"2024-09-12T01:33:19.485898+0000","last_fullsized":"2024-09-12T01:33:19.485898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:28.321407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218580+0000","last_change":"2024-09-12T01:33:12.173488+0000","last_active":"2024-09-12T01:33:18.218580+0000","last_peered":"2024-09-12T01:33:18.218580+0000","last_clean":"2024-09-12T01:33:18.218580+0000","last_became_active":"2024-09-12T01:33:12.173187+0000","last_became_peered":"2024-09-12T01:33:12.173187+0000","last_unstale":"2024-09-12T01:33:18.218580+0000","last_undegraded":"2024-09-12T01:33:18.218580+0000","last_fullsized":"2024-09-12T01:33:18.218580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:27.581552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485330+0000","last_change":"2024-09-12T01:33:11.149565+0000","last_active":"2024-09-12T01:33:19.485330+0000","last_peered":"2024-09-12T01:33:19.485330+0000","last_clean":"2024-09-12T01:33:19.485330+0000","last_became_active":"2024-09-12T01:33:11.149034+0000","last_became_peered":"2024-09-12T01:33:11.149034+0000","last_unstale":"2024-09-12T01:33:19.485330+0000","last_undegraded":"2024-09-12T01:33:19.485330+0000","last_fullsized":"2024-09-12T01:33:19.485330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218212+0000","last_change":"2024-09-12T01:33:16.202414+0000","last_active":"2024-09-12T01:33:18.218212+0000","last_peered":"2024-09-12T01:33:18.218212+0000","last_clean":"2024-09-12T01:33:18.218212+0000","last_became_active":"2024-09-12T01:33:16.200143+0000","last_became_peered":"2024-09-12T01:33:16.200143+0000","last_unstale":"2024-09-12T01:33:18.218212+0000","last_undegraded":"2024-09-12T01:33:18.218212+0000","last_fullsized":"2024-09-12T01:33:18.218212+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:14.088579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413595+0000","last_change":"2024-09-12T01:33:12.174901+0000","last_active":"2024-09-12T01:33:19.413595+0000","last_peered":"2024-09-12T01:33:19.413595+0000","last_clean":"2024-09-12T01:33:19.413595+0000","last_became_active":"2024-09-12T01:33:12.174638+0000","last_became_peered":"2024-09-12T01:33:12.174638+0000","last_unstale":"2024-09-12T01:33:19.413595+0000","last_undegraded":"2024-09-12T01:33:19.413595+0000","last_fullsized":"2024-09-12T01:33:19.413595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:54.169299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839752+0000","last_change":"2024-09-12T01:33:11.147387+0000","last_active":"2024-09-12T01:33:18.839752+0000","last_peered":"2024-09-12T01:33:18.839752+0000","last_clean":"2024-09-12T01:33:18.839752+0000","last_became_active":"2024-09-12T01:33:11.147110+0000","last_became_peered":"2024-09-12T01:33:11.147110+0000","last_unstale":"2024-09-12T01:33:18.839752+0000","last_undegraded":"2024-09-12T01:33:18.839752+0000","last_fullsized":"2024-09-12T01:33:18.839752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216657+0000","last_change":"2024-09-12T01:33:16.191800+0000","last_active":"2024-09-12T01:33:18.216657+0000","last_peered":"2024-09-12T01:33:18.216657+0000","last_clean":"2024-09-12T01:33:18.216657+0000","last_became_active":"2024-09-12T01:33:16.190448+0000","last_became_peered":"2024-09-12T01:33:16.190448+0000","last_unstale":"2024-09-12T01:33:18.216657+0000","last_undegraded":"2024-09-12T01:33:18.216657+0000","last_fullsized":"2024-09-12T01:33:18.216657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:47.099767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839819+0000","last_change":"2024-09-12T01:33:12.178000+0000","last_active":"2024-09-12T01:33:18.839819+0000","last_peered":"2024-09-12T01:33:18.839819+0000","last_clean":"2024-09-12T01:33:18.839819+0000","last_became_active":"2024-09-12T01:33:12.177703+0000","last_became_peered":"2024-09-12T01:33:12.177703+0000","last_unstale":"2024-09-12T01:33:18.839819+0000","last_undegraded":"2024-09-12T01:33:18.839819+0000","last_fullsized":"2024-09-12T01:33:18.839819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:56.959747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485907+0000","last_change":"2024-09-12T01:33:11.149688+0000","last_active":"2024-09-12T01:33:19.485907+0000","last_peered":"2024-09-12T01:33:19.485907+0000","last_clean":"2024-09-12T01:33:19.485907+0000","last_became_active":"2024-09-12T01:33:11.149363+0000","last_became_peered":"2024-09-12T01:33:11.149363+0000","last_unstale":"2024-09-12T01:33:19.485907+0000","last_undegraded":"2024-09-12T01:33:19.485907+0000","last_fullsized":"2024-09-12T01:33:19.485907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216625+0000","last_change":"2024-09-12T01:33:16.592951+0000","last_active":"2024-09-12T01:33:18.216625+0000","last_peered":"2024-09-12T01:33:18.216625+0000","last_clean":"2024-09-12T01:33:18.216625+0000","last_became_active":"2024-09-12T01:33:16.592558+0000","last_became_peered":"2024-09-12T01:33:16.592558+0000","last_unstale":"2024-09-12T01:33:18.216625+0000","last_undegraded":"2024-09-12T01:33:18.216625+0000","last_fullsized":"2024-09-12T01:33:18.216625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:29.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413385+0000","last_change":"2024-09-12T01:33:16.594318+0000","last_active":"2024-09-12T01:33:19.413385+0000","last_peered":"2024-09-12T01:33:19.413385+0000","last_clean":"2024-09-12T01:33:19.413385+0000","last_became_active":"2024-09-12T01:33:16.593834+0000","last_became_peered":"2024-09-12T01:33:16.593834+0000","last_unstale":"2024-09-12T01:33:19.413385+0000","last_undegraded":"2024-09-12T01:33:19.413385+0000","last_fullsized":"2024-09-12T01:33:19.413385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:11.126504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485462+0000","last_change":"2024-09-12T01:33:12.171282+0000","last_active":"2024-09-12T01:33:19.485462+0000","last_peered":"2024-09-12T01:33:19.485462+0000","last_clean":"2024-09-12T01:33:19.485462+0000","last_became_active":"2024-09-12T01:33:12.170988+0000","last_became_peered":"2024-09-12T01:33:12.170988+0000","last_unstale":"2024-09-12T01:33:19.485462+0000","last_undegraded":"2024-09-12T01:33:19.485462+0000","last_fullsized":"2024-09-12T01:33:19.485462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:25.007983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839177+0000","last_change":"2024-09-12T01:33:11.146497+0000","last_active":"2024-09-12T01:33:18.839177+0000","last_peered":"2024-09-12T01:33:18.839177+0000","last_clean":"2024-09-12T01:33:18.839177+0000","last_became_active":"2024-09-12T01:33:11.146184+0000","last_became_peered":"2024-09-12T01:33:11.146184+0000","last_unstale":"2024-09-12T01:33:18.839177+0000","last_undegraded":"2024-09-12T01:33:18.839177+0000","last_fullsized":"2024-09-12T01:33:18.839177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487513+0000","last_change":"2024-09-12T01:33:16.202584+0000","last_active":"2024-09-12T01:33:19.487513+0000","last_peered":"2024-09-12T01:33:19.487513+0000","last_clean":"2024-09-12T01:33:19.487513+0000","last_became_active":"2024-09-12T01:33:16.202137+0000","last_became_peered":"2024-09-12T01:33:16.202137+0000","last_unstale":"2024-09-12T01:33:19.487513+0000","last_undegraded":"2024-09-12T01:33:19.487513+0000","last_fullsized":"2024-09-12T01:33:19.487513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:39.155565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216613+0000","last_change":"2024-09-12T01:33:12.169183+0000","last_active":"2024-09-12T01:33:18.216613+0000","last_peered":"2024-09-12T01:33:18.216613+0000","last_clean":"2024-09-12T01:33:18.216613+0000","last_became_active":"2024-09-12T01:33:12.168803+0000","last_became_peered":"2024-09-12T01:33:12.168803+0000","last_unstale":"2024-09-12T01:33:18.216613+0000","last_undegraded":"2024-09-12T01:33:18.216613+0000","last_fullsized":"2024-09-12T01:33:18.216613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:25.007983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216934+0000","last_change":"2024-09-12T01:33:11.144264+0000","last_active":"2024-09-12T01:33:18.216934+0000","last_peered":"2024-09-12T01:33:18.216934+0000","last_clean":"2024-09-12T01:33:18.216934+0000","last_became_active":"2024-09-12T01:33:11.143993+0000","last_became_peered":"2024-09-12T01:33:11.143993+0000","last_unstale":"2024-09-12T01:33:18.216934+0000","last_undegraded":"2024-09-12T01:33:18.216934+0000","last_fullsized":"2024-09-12T01:33:18.216934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:02:33.335627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485661+0000","last_change":"2024-09-12T01:33:11.143410+0000","last_active":"2024-09-12T01:33:19.485661+0000","last_peered":"2024-09-12T01:33:19.485661+0000","last_clean":"2024-09-12T01:33:19.485661+0000","last_became_active":"2024-09-12T01:33:11.143121+0000","last_became_peered":"2024-09-12T01:33:11.143121+0000","last_unstale":"2024-09-12T01:33:19.485661+0000","last_undegraded":"2024-09-12T01:33:19.485661+0000","last_fullsized":"2024-09-12T01:33:19.485661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413493+0000","last_change":"2024-09-12T01:33:12.176891+0000","last_active":"2024-09-12T01:33:19.413493+0000","last_peered":"2024-09-12T01:33:19.413493+0000","last_clean":"2024-09-12T01:33:19.413493+0000","last_became_active":"2024-09-12T01:33:12.176573+0000","last_became_peered":"2024-09-12T01:33:12.176573+0000","last_unstale":"2024-09-12T01:33:19.413493+0000","last_undegraded":"2024-09-12T01:33:19.413493+0000","last_fullsized":"2024-09-12T01:33:19.413493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:47.157354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554126+0000","last_change":"2024-09-12T01:33:16.195123+0000","last_active":"2024-09-12T01:33:18.554126+0000","last_peered":"2024-09-12T01:33:18.554126+0000","last_clean":"2024-09-12T01:33:18.554126+0000","last_became_active":"2024-09-12T01:33:16.194690+0000","last_became_peered":"2024-09-12T01:33:16.194690+0000","last_unstale":"2024-09-12T01:33:18.554126+0000","last_undegraded":"2024-09-12T01:33:18.554126+0000","last_fullsized":"2024-09-12T01:33:18.554126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:14.296077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217027+0000","last_change":"2024-09-12T01:33:11.145695+0000","last_active":"2024-09-12T01:33:18.217027+0000","last_peered":"2024-09-12T01:33:18.217027+0000","last_clean":"2024-09-12T01:33:18.217027+0000","last_became_active":"2024-09-12T01:33:11.145391+0000","last_became_peered":"2024-09-12T01:33:11.145391+0000","last_unstale":"2024-09-12T01:33:18.217027+0000","last_undegraded":"2024-09-12T01:33:18.217027+0000","last_fullsized":"2024-09-12T01:33:18.217027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:47.695596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218383+0000","last_change":"2024-09-12T01:33:12.164156+0000","last_active":"2024-09-12T01:33:18.218383+0000","last_peered":"2024-09-12T01:33:18.218383+0000","last_clean":"2024-09-12T01:33:18.218383+0000","last_became_active":"2024-09-12T01:33:12.160798+0000","last_became_peered":"2024-09-12T01:33:12.160798+0000","last_unstale":"2024-09-12T01:33:18.218383+0000","last_undegraded":"2024-09-12T01:33:18.218383+0000","last_fullsized":"2024-09-12T01:33:18.218383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:56.394003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217262+0000","last_change":"2024-09-12T01:33:16.198377+0000","last_active":"2024-09-12T01:33:18.217262+0000","last_peered":"2024-09-12T01:33:18.217262+0000","last_clean":"2024-09-12T01:33:18.217262+0000","last_became_active":"2024-09-12T01:33:16.197913+0000","last_became_peered":"2024-09-12T01:33:16.197913+0000","last_unstale":"2024-09-12T01:33:18.217262+0000","last_undegraded":"2024-09-12T01:33:18.217262+0000","last_fullsized":"2024-09-12T01:33:18.217262+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:56:05.124457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486041+0000","last_change":"2024-09-12T01:33:12.176137+0000","last_active":"2024-09-12T01:33:19.486041+0000","last_peered":"2024-09-12T01:33:19.486041+0000","last_clean":"2024-09-12T01:33:19.486041+0000","last_became_active":"2024-09-12T01:33:12.175840+0000","last_became_peered":"2024-09-12T01:33:12.175840+0000","last_unstale":"2024-09-12T01:33:19.486041+0000","last_undegraded":"2024-09-12T01:33:19.486041+0000","last_fullsized":"2024-09-12T01:33:19.486041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:38.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554012+0000","last_change":"2024-09-12T01:33:12.168134+0000","last_active":"2024-09-12T01:33:18.554012+0000","last_peered":"2024-09-12T01:33:18.554012+0000","last_clean":"2024-09-12T01:33:18.554012+0000","last_became_active":"2024-09-12T01:33:12.167844+0000","last_became_peered":"2024-09-12T01:33:12.167844+0000","last_unstale":"2024-09-12T01:33:18.554012+0000","last_undegraded":"2024-09-12T01:33:18.554012+0000","last_fullsized":"2024-09-12T01:33:18.554012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:06.175990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839339+0000","last_change":"2024-09-12T01:33:12.183112+0000","last_active":"2024-09-12T01:33:18.839339+0000","last_peered":"2024-09-12T01:33:18.839339+0000","last_clean":"2024-09-12T01:33:18.839339+0000","last_became_active":"2024-09-12T01:33:12.182858+0000","last_became_peered":"2024-09-12T01:33:12.182858+0000","last_unstale":"2024-09-12T01:33:18.839339+0000","last_undegraded":"2024-09-12T01:33:18.839339+0000","last_fullsized":"2024-09-12T01:33:18.839339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:11.508027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413461+0000","last_change":"2024-09-12T01:33:12.181683+0000","last_active":"2024-09-12T01:33:19.413461+0000","last_peered":"2024-09-12T01:33:19.413461+0000","last_clean":"2024-09-12T01:33:19.413461+0000","last_became_active":"2024-09-12T01:33:12.181383+0000","last_became_peered":"2024-09-12T01:33:12.181383+0000","last_unstale":"2024-09-12T01:33:19.413461+0000","last_undegraded":"2024-09-12T01:33:19.413461+0000","last_fullsized":"2024-09-12T01:33:19.413461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:00.865640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217578+0000","last_change":"2024-09-12T01:33:12.178479+0000","last_active":"2024-09-12T01:33:18.217578+0000","last_peered":"2024-09-12T01:33:18.217578+0000","last_clean":"2024-09-12T01:33:18.217578+0000","last_became_active":"2024-09-12T01:33:12.178198+0000","last_became_peered":"2024-09-12T01:33:12.178198+0000","last_unstale":"2024-09-12T01:33:18.217578+0000","last_undegraded":"2024-09-12T01:33:18.217578+0000","last_fullsized":"2024-09-12T01:33:18.217578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:53.398489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485933+0000","last_change":"2024-09-12T01:33:12.168261+0000","last_active":"2024-09-12T01:33:19.485933+0000","last_peered":"2024-09-12T01:33:19.485933+0000","last_clean":"2024-09-12T01:33:19.485933+0000","last_became_active":"2024-09-12T01:33:12.167980+0000","last_became_peered":"2024-09-12T01:33:12.167980+0000","last_unstale":"2024-09-12T01:33:19.485933+0000","last_undegraded":"2024-09-12T01:33:19.485933+0000","last_fullsized":"2024-09-12T01:33:19.485933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:02.393918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487377+0000","last_change":"2024-09-12T01:33:12.186333+0000","last_active":"2024-09-12T01:33:19.487377+0000","last_peered":"2024-09-12T01:33:19.487377+0000","last_clean":"2024-09-12T01:33:19.487377+0000","last_became_active":"2024-09-12T01:33:12.186047+0000","last_became_peered":"2024-09-12T01:33:12.186047+0000","last_unstale":"2024-09-12T01:33:19.487377+0000","last_undegraded":"2024-09-12T01:33:19.487377+0000","last_fullsized":"2024-09-12T01:33:19.487377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:53.398489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217203+0000","last_change":"2024-09-12T01:33:12.177911+0000","last_active":"2024-09-12T01:33:18.217203+0000","last_peered":"2024-09-12T01:33:18.217203+0000","last_clean":"2024-09-12T01:33:18.217203+0000","last_became_active":"2024-09-12T01:33:12.177625+0000","last_became_peered":"2024-09-12T01:33:12.177625+0000","last_unstale":"2024-09-12T01:33:18.217203+0000","last_undegraded":"2024-09-12T01:33:18.217203+0000","last_fullsized":"2024-09-12T01:33:18.217203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:02.393918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839173+0000","last_change":"2024-09-12T01:33:12.183935+0000","last_active":"2024-09-12T01:33:18.839173+0000","last_peered":"2024-09-12T01:33:18.839173+0000","last_clean":"2024-09-12T01:33:18.839173+0000","last_became_active":"2024-09-12T01:33:12.183620+0000","last_became_peered":"2024-09-12T01:33:12.183620+0000","last_unstale":"2024-09-12T01:33:18.839173+0000","last_undegraded":"2024-09-12T01:33:18.839173+0000","last_fullsized":"2024-09-12T01:33:18.839173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:02.950225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216744+0000","last_change":"2024-09-12T01:33:12.179130+0000","last_active":"2024-09-12T01:33:18.216744+0000","last_peered":"2024-09-12T01:33:18.216744+0000","last_clean":"2024-09-12T01:33:18.216744+0000","last_became_active":"2024-09-12T01:33:12.177141+0000","last_became_peered":"2024-09-12T01:33:12.177141+0000","last_unstale":"2024-09-12T01:33:18.216744+0000","last_undegraded":"2024-09-12T01:33:18.216744+0000","last_fullsized":"2024-09-12T01:33:18.216744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:24.134321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839306+0000","last_change":"2024-09-12T01:33:12.175899+0000","last_active":"2024-09-12T01:33:18.839306+0000","last_peered":"2024-09-12T01:33:18.839306+0000","last_clean":"2024-09-12T01:33:18.839306+0000","last_became_active":"2024-09-12T01:33:12.175594+0000","last_became_peered":"2024-09-12T01:33:12.175594+0000","last_unstale":"2024-09-12T01:33:18.839306+0000","last_undegraded":"2024-09-12T01:33:18.839306+0000","last_fullsized":"2024-09-12T01:33:18.839306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:51.254891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413194+0000","last_change":"2024-09-12T01:33:12.185737+0000","last_active":"2024-09-12T01:33:19.413194+0000","last_peered":"2024-09-12T01:33:19.413194+0000","last_clean":"2024-09-12T01:33:19.413194+0000","last_became_active":"2024-09-12T01:33:12.185489+0000","last_became_peered":"2024-09-12T01:33:12.185489+0000","last_unstale":"2024-09-12T01:33:19.413194+0000","last_undegraded":"2024-09-12T01:33:19.413194+0000","last_fullsized":"2024-09-12T01:33:19.413194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:05.301039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486993+0000","last_change":"2024-09-12T01:33:12.183324+0000","last_active":"2024-09-12T01:33:19.486993+0000","last_peered":"2024-09-12T01:33:19.486993+0000","last_clean":"2024-09-12T01:33:19.486993+0000","last_became_active":"2024-09-12T01:33:12.182857+0000","last_became_peered":"2024-09-12T01:33:12.182857+0000","last_unstale":"2024-09-12T01:33:19.486993+0000","last_undegraded":"2024-09-12T01:33:19.486993+0000","last_fullsized":"2024-09-12T01:33:19.486993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:52.439908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487522+0000","last_change":"2024-09-12T01:33:12.177539+0000","last_active":"2024-09-12T01:33:19.487522+0000","last_peered":"2024-09-12T01:33:19.487522+0000","last_clean":"2024-09-12T01:33:19.487522+0000","last_became_active":"2024-09-12T01:33:12.173980+0000","last_became_peered":"2024-09-12T01:33:12.173980+0000","last_unstale":"2024-09-12T01:33:19.487522+0000","last_undegraded":"2024-09-12T01:33:19.487522+0000","last_fullsized":"2024-09-12T01:33:19.487522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:11.508027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554334+0000","last_change":"2024-09-12T01:33:12.179198+0000","last_active":"2024-09-12T01:33:18.554334+0000","last_peered":"2024-09-12T01:33:18.554334+0000","last_clean":"2024-09-12T01:33:18.554334+0000","last_became_active":"2024-09-12T01:33:12.178859+0000","last_became_peered":"2024-09-12T01:33:12.178859+0000","last_unstale":"2024-09-12T01:33:18.554334+0000","last_undegraded":"2024-09-12T01:33:18.554334+0000","last_fullsized":"2024-09-12T01:33:18.554334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:46.228922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839997+0000","last_change":"2024-09-12T01:33:12.179001+0000","last_active":"2024-09-12T01:33:18.839997+0000","last_peered":"2024-09-12T01:33:18.839997+0000","last_clean":"2024-09-12T01:33:18.839997+0000","last_became_active":"2024-09-12T01:33:12.178298+0000","last_became_peered":"2024-09-12T01:33:12.178298+0000","last_unstale":"2024-09-12T01:33:18.839997+0000","last_undegraded":"2024-09-12T01:33:18.839997+0000","last_fullsized":"2024-09-12T01:33:18.839997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:35.107896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839003+0000","last_change":"2024-09-12T01:33:12.169863+0000","last_active":"2024-09-12T01:33:18.839003+0000","last_peered":"2024-09-12T01:33:18.839003+0000","last_clean":"2024-09-12T01:33:18.839003+0000","last_became_active":"2024-09-12T01:33:12.169441+0000","last_became_peered":"2024-09-12T01:33:12.169441+0000","last_unstale":"2024-09-12T01:33:18.839003+0000","last_undegraded":"2024-09-12T01:33:18.839003+0000","last_fullsized":"2024-09-12T01:33:18.839003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485524+0000","last_change":"2024-09-12T01:33:12.160854+0000","last_active":"2024-09-12T01:33:19.485524+0000","last_peered":"2024-09-12T01:33:19.485524+0000","last_clean":"2024-09-12T01:33:19.485524+0000","last_became_active":"2024-09-12T01:33:12.160512+0000","last_became_peered":"2024-09-12T01:33:12.160512+0000","last_unstale":"2024-09-12T01:33:19.485524+0000","last_undegraded":"2024-09-12T01:33:19.485524+0000","last_fullsized":"2024-09-12T01:33:19.485524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:38.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413165+0000","last_change":"2024-09-12T01:33:12.172031+0000","last_active":"2024-09-12T01:33:19.413165+0000","last_peered":"2024-09-12T01:33:19.413165+0000","last_clean":"2024-09-12T01:33:19.413165+0000","last_became_active":"2024-09-12T01:33:12.171730+0000","last_became_peered":"2024-09-12T01:33:12.171730+0000","last_unstale":"2024-09-12T01:33:19.413165+0000","last_undegraded":"2024-09-12T01:33:19.413165+0000","last_fullsized":"2024-09-12T01:33:19.413165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:10.248408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216814+0000","last_change":"2024-09-12T01:33:12.173718+0000","last_active":"2024-09-12T01:33:18.216814+0000","last_peered":"2024-09-12T01:33:18.216814+0000","last_clean":"2024-09-12T01:33:18.216814+0000","last_became_active":"2024-09-12T01:33:12.171379+0000","last_became_peered":"2024-09-12T01:33:12.171379+0000","last_unstale":"2024-09-12T01:33:18.216814+0000","last_undegraded":"2024-09-12T01:33:18.216814+0000","last_fullsized":"2024-09-12T01:33:18.216814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:52.439908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218338+0000","last_change":"2024-09-12T01:33:12.174895+0000","last_active":"2024-09-12T01:33:18.218338+0000","last_peered":"2024-09-12T01:33:18.218338+0000","last_clean":"2024-09-12T01:33:18.218338+0000","last_became_active":"2024-09-12T01:33:12.174591+0000","last_became_peered":"2024-09-12T01:33:12.174591+0000","last_unstale":"2024-09-12T01:33:18.218338+0000","last_undegraded":"2024-09-12T01:33:18.218338+0000","last_fullsized":"2024-09-12T01:33:18.218338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:57.102634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486963+0000","last_change":"2024-09-12T01:33:12.181588+0000","last_active":"2024-09-12T01:33:19.486963+0000","last_peered":"2024-09-12T01:33:19.486963+0000","last_clean":"2024-09-12T01:33:19.486963+0000","last_became_active":"2024-09-12T01:33:12.181293+0000","last_became_peered":"2024-09-12T01:33:12.181293+0000","last_unstale":"2024-09-12T01:33:19.486963+0000","last_undegraded":"2024-09-12T01:33:19.486963+0000","last_fullsized":"2024-09-12T01:33:19.486963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:02.393918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487308+0000","last_change":"2024-09-12T01:33:12.175792+0000","last_active":"2024-09-12T01:33:19.487308+0000","last_peered":"2024-09-12T01:33:19.487308+0000","last_clean":"2024-09-12T01:33:19.487308+0000","last_became_active":"2024-09-12T01:33:12.175332+0000","last_became_peered":"2024-09-12T01:33:12.175332+0000","last_unstale":"2024-09-12T01:33:19.487308+0000","last_undegraded":"2024-09-12T01:33:19.487308+0000","last_fullsized":"2024-09-12T01:33:19.487308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487796+0000","last_change":"2024-09-12T01:33:12.178831+0000","last_active":"2024-09-12T01:33:19.487796+0000","last_peered":"2024-09-12T01:33:19.487796+0000","last_clean":"2024-09-12T01:33:19.487796+0000","last_became_active":"2024-09-12T01:33:12.178116+0000","last_became_peered":"2024-09-12T01:33:12.178116+0000","last_unstale":"2024-09-12T01:33:19.487796+0000","last_undegraded":"2024-09-12T01:33:19.487796+0000","last_fullsized":"2024-09-12T01:33:19.487796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:35.107896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216514+0000","last_change":"2024-09-12T01:33:12.172225+0000","last_active":"2024-09-12T01:33:18.216514+0000","last_peered":"2024-09-12T01:33:18.216514+0000","last_clean":"2024-09-12T01:33:18.216514+0000","last_became_active":"2024-09-12T01:33:12.171933+0000","last_became_peered":"2024-09-12T01:33:12.171933+0000","last_unstale":"2024-09-12T01:33:18.216514+0000","last_undegraded":"2024-09-12T01:33:18.216514+0000","last_fullsized":"2024-09-12T01:33:18.216514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:20.867318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486602+0000","last_change":"2024-09-12T01:33:12.173395+0000","last_active":"2024-09-12T01:33:19.486602+0000","last_peered":"2024-09-12T01:33:19.486602+0000","last_clean":"2024-09-12T01:33:19.486602+0000","last_became_active":"2024-09-12T01:33:12.173090+0000","last_became_peered":"2024-09-12T01:33:12.173090+0000","last_unstale":"2024-09-12T01:33:19.486602+0000","last_undegraded":"2024-09-12T01:33:19.486602+0000","last_fullsized":"2024-09-12T01:33:19.486602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:46.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839213+0000","last_change":"2024-09-12T01:33:12.169604+0000","last_active":"2024-09-12T01:33:18.839213+0000","last_peered":"2024-09-12T01:33:18.839213+0000","last_clean":"2024-09-12T01:33:18.839213+0000","last_became_active":"2024-09-12T01:33:12.169258+0000","last_became_peered":"2024-09-12T01:33:12.169258+0000","last_unstale":"2024-09-12T01:33:18.839213+0000","last_undegraded":"2024-09-12T01:33:18.839213+0000","last_fullsized":"2024-09-12T01:33:18.839213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:46.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486571+0000","last_change":"2024-09-12T01:33:12.171033+0000","last_active":"2024-09-12T01:33:19.486571+0000","last_peered":"2024-09-12T01:33:19.486571+0000","last_clean":"2024-09-12T01:33:19.486571+0000","last_became_active":"2024-09-12T01:33:12.170708+0000","last_became_peered":"2024-09-12T01:33:12.170708+0000","last_unstale":"2024-09-12T01:33:19.486571+0000","last_undegraded":"2024-09-12T01:33:19.486571+0000","last_fullsized":"2024-09-12T01:33:19.486571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:56.959747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839539+0000","last_change":"2024-09-12T01:33:12.182780+0000","last_active":"2024-09-12T01:33:18.839539+0000","last_peered":"2024-09-12T01:33:18.839539+0000","last_clean":"2024-09-12T01:33:18.839539+0000","last_became_active":"2024-09-12T01:33:12.182454+0000","last_became_peered":"2024-09-12T01:33:12.182454+0000","last_unstale":"2024-09-12T01:33:18.839539+0000","last_undegraded":"2024-09-12T01:33:18.839539+0000","last_fullsized":"2024-09-12T01:33:18.839539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:53.398489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413295+0000","last_change":"2024-09-12T01:33:12.180723+0000","last_active":"2024-09-12T01:33:19.413295+0000","last_peered":"2024-09-12T01:33:19.413295+0000","last_clean":"2024-09-12T01:33:19.413295+0000","last_became_active":"2024-09-12T01:33:12.180414+0000","last_became_peered":"2024-09-12T01:33:12.180414+0000","last_unstale":"2024-09-12T01:33:19.413295+0000","last_undegraded":"2024-09-12T01:33:19.413295+0000","last_fullsized":"2024-09-12T01:33:19.413295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:58.901902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486128+0000","last_change":"2024-09-12T01:33:12.165745+0000","last_active":"2024-09-12T01:33:19.486128+0000","last_peered":"2024-09-12T01:33:19.486128+0000","last_clean":"2024-09-12T01:33:19.486128+0000","last_became_active":"2024-09-12T01:33:12.165475+0000","last_became_peered":"2024-09-12T01:33:12.165475+0000","last_unstale":"2024-09-12T01:33:19.486128+0000","last_undegraded":"2024-09-12T01:33:19.486128+0000","last_fullsized":"2024-09-12T01:33:19.486128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487685+0000","last_change":"2024-09-12T01:33:12.179105+0000","last_active":"2024-09-12T01:33:19.487685+0000","last_peered":"2024-09-12T01:33:19.487685+0000","last_clean":"2024-09-12T01:33:19.487685+0000","last_became_active":"2024-09-12T01:33:12.173779+0000","last_became_peered":"2024-09-12T01:33:12.173779+0000","last_unstale":"2024-09-12T01:33:19.487685+0000","last_undegraded":"2024-09-12T01:33:19.487685+0000","last_fullsized":"2024-09-12T01:33:19.487685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:02.950225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":700890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":113784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":110030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":110066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":282624,"data_stored":113784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":110100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":270336,"data_stored":109940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":876544,"data_stored":700946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:33:32.599 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T01:33:32.600 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T01:33:32.600 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-12T01:33:32.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-12T01:33:32.600 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-12T01:33:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.737+0000 7f0ce9053640 1 Processor -- start 2024-09-12T01:33:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.737+0000 7f0ce9053640 1 -- start start 2024-09-12T01:33:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.738+0000 7f0ce9053640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0ce415e510 0x7f0ce40ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.738+0000 7f0ce9053640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ce40d3430 con 0x7f0ce40d13d0 2024-09-12T01:33:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.738+0000 7f0ce9053640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ce40d36d0 con 0x7f0ce40cf040 2024-09-12T01:33:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.738+0000 7f0ce9053640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f0ce40d39b0 con 0x7f0ce415e510 2024-09-12T01:33:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.738+0000 7f0ce359e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0ce40d13d0 0x7f0ce40b78e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53594/0 (socket says 172.21.15.16:53594) 2024-09-12T01:33:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.738+0000 7f0ce359e640 1 -- 172.21.15.16:0/723377973 learned_addr learned my addr 172.21.15.16:0/723377973 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.738+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1767432870 0 0) 0x7f0ce40d3430 con 0x7f0ce40d13d0 2024-09-12T01:33:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ccc003520 con 0x7f0ce40d13d0 2024-09-12T01:33:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce2d9d640 1 --2- 172.21.15.16:0/723377973 >> v2:172.21.15.33:3300/0 conn(0x7f0ce415e510 0x7f0ce40ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1808280365 0 0) 0x7f0ccc003520 con 0x7f0ce40d13d0 2024-09-12T01:33:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 >> v2:172.21.15.33:3300/0 conn(0x7f0ce415e510 msgr2=0x7f0ce40ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 --2- 172.21.15.16:0/723377973 >> v2:172.21.15.33:3300/0 conn(0x7f0ce415e510 0x7f0ce40ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 >> v1:172.21.15.33:6789/0 conn(0x7f0ce40cf040 legacy=0x7f0ce40d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce40d3c90 con 0x7f0ce40d13d0 2024-09-12T01:33:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f0cdc0019f0 con 0x7f0ce40d13d0 2024-09-12T01:33:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 >> v1:172.21.15.16:6789/0 conn(0x7f0ce40d13d0 legacy=0x7f0ce40b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 --2- 172.21.15.16:0/723377973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ccc003a20 0x7f0ccc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 --2- 172.21.15.16:0/723377973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ccc004800 0x7f0ccc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 --2- 172.21.15.16:0/723377973 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0ccc0051c0 0x7f0ccc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce40d39b0 con 0x7f0ccc004800 2024-09-12T01:33:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc00c0f0 con 0x7f0ccc003a20 2024-09-12T01:33:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0ccc00c2f0 con 0x7f0ccc0051c0 2024-09-12T01:33:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce259c640 1 --2- 172.21.15.16:0/723377973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ccc004800 0x7f0ccc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce2d9d640 1 --2- 172.21.15.16:0/723377973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ccc003a20 0x7f0ccc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce359e640 1 --2- 172.21.15.16:0/723377973 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0ccc0051c0 0x7f0ccc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce259c640 1 -- 172.21.15.16:0/723377973 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0ccc0051c0 msgr2=0x7f0ccc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce259c640 1 --2- 172.21.15.16:0/723377973 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0ccc0051c0 0x7f0ccc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce259c640 1 -- 172.21.15.16:0/723377973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ccc003a20 msgr2=0x7f0ccc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce259c640 1 --2- 172.21.15.16:0/723377973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ccc003a20 0x7f0ccc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce259c640 1 -- 172.21.15.16:0/723377973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc003520 con 0x7f0ccc004800 2024-09-12T01:33:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.739+0000 7f0ce359e640 1 -- 172.21.15.16:0/723377973 reap_dead start 2024-09-12T01:33:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.740+0000 7f0ce259c640 1 --2- 172.21.15.16:0/723377973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ccc004800 0x7f0ccc004c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8001860 tx=0x7f0ccc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15451a9c82d10c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.740+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8037070 con 0x7f0ccc004800 2024-09-12T01:33:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.740+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd802abe0 con 0x7f0ccc004800 2024-09-12T01:33:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.740+0000 7f0ce1d9b640 1 -- 172.21.15.16:0/723377973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8001430 con 0x7f0ccc004800 2024-09-12T01:33:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.741+0000 7f0ce9053640 1 -- 172.21.15.16:0/723377973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ccc004800 msgr2=0x7f0ccc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.741+0000 7f0ce9053640 1 --2- 172.21.15.16:0/723377973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ccc004800 0x7f0ccc004c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8001860 tx=0x7f0ccc005830 comp rx=0 tx=0).stop 2024-09-12T01:33:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.741+0000 7f0ce9053640 1 -- 172.21.15.16:0/723377973 shutdown_connections 2024-09-12T01:33:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.741+0000 7f0ce9053640 1 --2- 172.21.15.16:0/723377973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ccc004800 0x7f0ccc004c80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.741+0000 7f0ce9053640 1 -- 172.21.15.16:0/723377973 >> 172.21.15.16:0/723377973 conn(0x7f0ce406d1a0 msgr2=0x7f0ce4073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.741+0000 7f0ce9053640 1 -- 172.21.15.16:0/723377973 shutdown_connections 2024-09-12T01:33:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.741+0000 7f0ce9053640 1 -- 172.21.15.16:0/723377973 wait complete. 2024-09-12T01:33:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.742+0000 7f0ce9053640 1 Processor -- start 2024-09-12T01:33:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.742+0000 7f0ce9053640 1 -- start start 2024-09-12T01:33:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.743+0000 7f0ce9053640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0ccc004800 0x7f0ce414b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.743+0000 7f0ce9053640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce4157280 0x7f0ce415ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.743+0000 7f0ce9053640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce4157700 0x7f0ce415cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.743+0000 7f0ce9053640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce415e8f0 con 0x7f0ce4157280 2024-09-12T01:33:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.743+0000 7f0ce9053640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce40d22a0 con 0x7f0ce4157700 2024-09-12T01:33:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.743+0000 7f0ce9053640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0ce415ea70 con 0x7f0ccc004800 2024-09-12T01:33:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.743+0000 7f0ce259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce4157280 0x7f0ce415ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.743+0000 7f0ce259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce4157280 0x7f0ce415ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42062/0 (socket says 172.21.15.16:42062) 2024-09-12T01:33:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.743+0000 7f0ce259c640 1 -- 172.21.15.16:0/2767067417 learned_addr learned my addr 172.21.15.16:0/2767067417 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.743+0000 7f0ce2d9d640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0ccc004800 0x7f0ce414b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.743+0000 7f0ce359e640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce4157700 0x7f0ce415cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.744+0000 7f0ce259c640 1 -- 172.21.15.16:0/2767067417 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0ccc004800 msgr2=0x7f0ce414b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.744+0000 7f0ce259c640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0ccc004800 0x7f0ce414b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.744+0000 7f0ce259c640 1 -- 172.21.15.16:0/2767067417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce4157700 msgr2=0x7f0ce415cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.744+0000 7f0ce259c640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce4157700 0x7f0ce415cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.744+0000 7f0ce259c640 1 -- 172.21.15.16:0/2767067417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce40b5200 con 0x7f0ce4157280 2024-09-12T01:33:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.744+0000 7f0ce259c640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce4157280 0x7f0ce415ca80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8001950 tx=0x7f0cd8001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.744+0000 7f0ccb7fe640 1 -- 172.21.15.16:0/2767067417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8037050 con 0x7f0ce4157280 2024-09-12T01:33:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.744+0000 7f0ccb7fe640 1 -- 172.21.15.16:0/2767067417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd8043070 con 0x7f0ce4157280 2024-09-12T01:33:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.744+0000 7f0ce9053640 1 -- 172.21.15.16:0/2767067417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce40b4160 con 0x7f0ce4157280 2024-09-12T01:33:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.744+0000 7f0ccb7fe640 1 -- 172.21.15.16:0/2767067417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8001430 con 0x7f0ce4157280 2024-09-12T01:33:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.744+0000 7f0ce9053640 1 -- 172.21.15.16:0/2767067417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce40b4720 con 0x7f0ce4157280 2024-09-12T01:33:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.745+0000 7f0ce9053640 1 -- 172.21.15.16:0/2767067417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce4154f50 con 0x7f0ce4157280 2024-09-12T01:33:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.745+0000 7f0ccb7fe640 1 -- 172.21.15.16:0/2767067417 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0cd803e890 con 0x7f0ce4157280 2024-09-12T01:33:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.746+0000 7f0ccb7fe640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0cd006be30 0x7f0cd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.746+0000 7f0ccb7fe640 1 -- 172.21.15.16:0/2767067417 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0cd80ab8c0 con 0x7f0ce4157280 2024-09-12T01:33:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.746+0000 7f0ce2d9d640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0cd006be30 0x7f0cd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.746+0000 7f0ce2d9d640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0cd006be30 0x7f0cd006e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009bc0 tx=0x7f0cd4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.750+0000 7f0ccb7fe640 1 -- 172.21.15.16:0/2767067417 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0cd804d020 con 0x7f0ce4157280 2024-09-12T01:33:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.971+0000 7f0ce9053640 1 -- 172.21.15.16:0/2767067417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0ce415a4b0 con 0x7f0ce4157280 2024-09-12T01:33:32.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.974+0000 7f0ccb7fe640 1 -- 172.21.15.16:0/2767067417 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172938 (secure 0 0 0) 0x7f0cd8079a10 con 0x7f0ce4157280 2024-09-12T01:33:32.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.977+0000 7f0ce9053640 1 -- 172.21.15.16:0/2767067417 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0cd006be30 msgr2=0x7f0cd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.977+0000 7f0ce9053640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0cd006be30 0x7f0cd006e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009bc0 tx=0x7f0cd4009190 comp rx=0 tx=0).stop 2024-09-12T01:33:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.977+0000 7f0ce9053640 1 -- 172.21.15.16:0/2767067417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce4157280 msgr2=0x7f0ce415ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.977+0000 7f0ce9053640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce4157280 0x7f0ce415ca80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8001950 tx=0x7f0cd8001230 comp rx=0 tx=0).stop 2024-09-12T01:33:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.977+0000 7f0ce9053640 1 -- 172.21.15.16:0/2767067417 shutdown_connections 2024-09-12T01:33:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.977+0000 7f0ce9053640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0cd006be30 0x7f0cd006e2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.977+0000 7f0ce9053640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ce4157700 0x7f0ce415cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.977+0000 7f0ce9053640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce4157280 0x7f0ce415ca80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.977+0000 7f0ce9053640 1 --2- 172.21.15.16:0/2767067417 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0ccc004800 0x7f0ce414b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.977+0000 7f0ce9053640 1 -- 172.21.15.16:0/2767067417 >> 172.21.15.16:0/2767067417 conn(0x7f0ce406d1a0 msgr2=0x7f0ce40d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.977+0000 7f0ce9053640 1 -- 172.21.15.16:0/2767067417 shutdown_connections 2024-09-12T01:33:32.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:32.977+0000 7f0ce9053640 1 -- 172.21.15.16:0/2767067417 wait complete. 2024-09-12T01:33:32.988 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2688455607},{"type":"v1","addr":"172.21.15.16:6833","nonce":2688455607}]},"active_addr":"172.21.15.16:6833/2688455607","active_change":"2024-09-12T01:32:52.462164+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4112,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":771324269}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3617576358}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3313628870}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3779396036}]}]} 2024-09-12T01:33:32.993 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-12T01:33:32.993 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-12T01:33:32.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:33:33.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.095+0000 7f8a18311640 1 Processor -- start 2024-09-12T01:33:33.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.096+0000 7f8a18311640 1 -- start start 2024-09-12T01:33:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.096+0000 7f8a18311640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8a1015e510 0x7f8a100ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.096+0000 7f8a18311640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8a100d13d0 0x7f8a100b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.096+0000 7f8a18311640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a100d3430 con 0x7f8a100cf040 2024-09-12T01:33:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.097+0000 7f8a18311640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f8a100d36d0 con 0x7f8a1015e510 2024-09-12T01:33:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.097+0000 7f8a18311640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f8a100d39b0 con 0x7f8a100d13d0 2024-09-12T01:33:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.097+0000 7f8a158ac640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8a100cf040 0x7f8a100d0cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53604/0 (socket says 172.21.15.16:53604) 2024-09-12T01:33:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.097+0000 7f8a158ac640 1 -- 172.21.15.16:0/2158740589 learned_addr learned my addr 172.21.15.16:0/2158740589 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.097+0000 7f8a168ae640 1 --2- 172.21.15.16:0/2158740589 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8a100d13d0 0x7f8a100b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.097+0000 7f8a160ad640 1 --2- 172.21.15.16:0/2158740589 >> v2:172.21.15.33:3300/0 conn(0x7f8a1015e510 0x7f8a100ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.097+0000 7f8a150ab640 1 -- 172.21.15.16:0/2158740589 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4210092646 0 0) 0x7f8a100d3430 con 0x7f8a100cf040 2024-09-12T01:33:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.097+0000 7f8a150ab640 1 -- 172.21.15.16:0/2158740589 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89fc003520 con 0x7f8a100cf040 2024-09-12T01:33:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.097+0000 7f8a168ae640 1 -- 172.21.15.16:0/2158740589 >> v2:172.21.15.33:3300/0 conn(0x7f8a1015e510 msgr2=0x7f8a100ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.097+0000 7f8a168ae640 1 --2- 172.21.15.16:0/2158740589 >> v2:172.21.15.33:3300/0 conn(0x7f8a1015e510 0x7f8a100ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.097+0000 7f8a168ae640 1 -- 172.21.15.16:0/2158740589 >> v1:172.21.15.16:6789/0 conn(0x7f8a100cf040 legacy=0x7f8a100d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.097+0000 7f8a168ae640 1 -- 172.21.15.16:0/2158740589 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a100d3c90 con 0x7f8a100d13d0 2024-09-12T01:33:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.098+0000 7f8a168ae640 1 --2- 172.21.15.16:0/2158740589 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8a100d13d0 0x7f8a100b78e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009d90 tx=0x7f8a0802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9283808524c99746 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.098+0000 7f8a150ab640 1 -- 172.21.15.16:0/2158740589 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08031600 con 0x7f8a100d13d0 2024-09-12T01:33:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.098+0000 7f8a150ab640 1 -- 172.21.15.16:0/2158740589 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a080317a0 con 0x7f8a100d13d0 2024-09-12T01:33:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.098+0000 7f8a150ab640 1 -- 172.21.15.16:0/2158740589 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08031b50 con 0x7f8a100d13d0 2024-09-12T01:33:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.099+0000 7f8a18311640 1 -- 172.21.15.16:0/2158740589 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8a100d13d0 msgr2=0x7f8a100b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.099+0000 7f8a18311640 1 --2- 172.21.15.16:0/2158740589 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8a100d13d0 0x7f8a100b78e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009d90 tx=0x7f8a0802a5d0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.099+0000 7f8a18311640 1 -- 172.21.15.16:0/2158740589 shutdown_connections 2024-09-12T01:33:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.099+0000 7f8a18311640 1 --2- 172.21.15.16:0/2158740589 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8a100d13d0 0x7f8a100b78e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.099+0000 7f8a18311640 1 --2- 172.21.15.16:0/2158740589 >> v2:172.21.15.33:3300/0 conn(0x7f8a1015e510 0x7f8a100ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.099+0000 7f8a18311640 1 -- 172.21.15.16:0/2158740589 >> 172.21.15.16:0/2158740589 conn(0x7f8a1006d1a0 msgr2=0x7f8a10073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.099+0000 7f8a18311640 1 -- 172.21.15.16:0/2158740589 shutdown_connections 2024-09-12T01:33:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.099+0000 7f8a18311640 1 -- 172.21.15.16:0/2158740589 wait complete. 2024-09-12T01:33:33.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.100+0000 7f8a18311640 1 Processor -- start 2024-09-12T01:33:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.100+0000 7f8a18311640 1 -- start start 2024-09-12T01:33:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a18311640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a100cf040 0x7f8a100b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a18311640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a100d13d0 0x7f8a1014fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a18311640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8a1015e510 0x7f8a101f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a18311640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1015e8f0 con 0x7f8a100d13d0 2024-09-12T01:33:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a18311640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a100d22a0 con 0x7f8a100cf040 2024-09-12T01:33:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a18311640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1015ea70 con 0x7f8a1015e510 2024-09-12T01:33:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a158ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a100d13d0 0x7f8a1014fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a158ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a100d13d0 0x7f8a1014fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42066/0 (socket says 172.21.15.16:42066) 2024-09-12T01:33:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a158ac640 1 -- 172.21.15.16:0/1665653090 learned_addr learned my addr 172.21.15.16:0/1665653090 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a160ad640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a100cf040 0x7f8a100b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a168ae640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8a1015e510 0x7f8a101f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a158ac640 1 -- 172.21.15.16:0/1665653090 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8a1015e510 msgr2=0x7f8a101f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a158ac640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8a1015e510 0x7f8a101f4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a158ac640 1 -- 172.21.15.16:0/1665653090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a100cf040 msgr2=0x7f8a100b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a158ac640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a100cf040 0x7f8a100b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a158ac640 1 -- 172.21.15.16:0/1665653090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a101f4fb0 con 0x7f8a100d13d0 2024-09-12T01:33:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a158ac640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a100d13d0 0x7f8a1014fc90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000c430 tx=0x7f8a0000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.101+0000 7f8a06ffd640 1 -- 172.21.15.16:0/1665653090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00017070 con 0x7f8a100d13d0 2024-09-12T01:33:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.102+0000 7f8a06ffd640 1 -- 172.21.15.16:0/1665653090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a00005ce0 con 0x7f8a100d13d0 2024-09-12T01:33:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.102+0000 7f8a18311640 1 -- 172.21.15.16:0/1665653090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1015f910 con 0x7f8a100d13d0 2024-09-12T01:33:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.102+0000 7f8a06ffd640 1 -- 172.21.15.16:0/1665653090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000072f0 con 0x7f8a100d13d0 2024-09-12T01:33:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.102+0000 7f8a18311640 1 -- 172.21.15.16:0/1665653090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a100cc530 con 0x7f8a100d13d0 2024-09-12T01:33:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.102+0000 7f8a18311640 1 -- 172.21.15.16:0/1665653090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a100ceb00 con 0x7f8a100d13d0 2024-09-12T01:33:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.103+0000 7f8a06ffd640 1 -- 172.21.15.16:0/1665653090 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a0001a070 con 0x7f8a100d13d0 2024-09-12T01:33:33.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.104+0000 7f8a06ffd640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f89f406be30 0x7f89f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.104+0000 7f8a160ad640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f89f406be30 0x7f89f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.104+0000 7f8a06ffd640 1 -- 172.21.15.16:0/1665653090 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8a0008a5c0 con 0x7f8a100d13d0 2024-09-12T01:33:33.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.104+0000 7f8a160ad640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f89f406be30 0x7f89f406e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a100ce170 tx=0x7f8a0c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.108+0000 7f8a06ffd640 1 -- 172.21.15.16:0/1665653090 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8a000585e0 con 0x7f8a100d13d0 2024-09-12T01:33:33.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.286+0000 7f8a18311640 1 -- 172.21.15.16:0/1665653090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a100cf740 con 0x7f8a100d13d0 2024-09-12T01:33:33.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.287+0000 7f8a06ffd640 1 -- 172.21.15.16:0/1665653090 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18574 (secure 0 0 0) 0x7f8a0005c8e0 con 0x7f8a100d13d0 2024-09-12T01:33:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:33.287 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:33:18.213622+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:33:09.364639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:33:10.478237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:33:14.204624+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:33:33.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.289+0000 7f8a18311640 1 -- 172.21.15.16:0/1665653090 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f89f406be30 msgr2=0x7f89f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.289+0000 7f8a18311640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f89f406be30 0x7f89f406e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a100ce170 tx=0x7f8a0c009190 comp rx=0 tx=0).stop 2024-09-12T01:33:33.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.289+0000 7f8a18311640 1 -- 172.21.15.16:0/1665653090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a100d13d0 msgr2=0x7f8a1014fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.289+0000 7f8a18311640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a100d13d0 0x7f8a1014fc90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000c430 tx=0x7f8a0000c8f0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.289+0000 7f8a18311640 1 -- 172.21.15.16:0/1665653090 shutdown_connections 2024-09-12T01:33:33.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.289+0000 7f8a18311640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f89f406be30 0x7f89f406e2f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.289+0000 7f8a18311640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8a1015e510 0x7f8a101f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.289+0000 7f8a18311640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a100d13d0 0x7f8a1014fc90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.289+0000 7f8a18311640 1 --2- 172.21.15.16:0/1665653090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a100cf040 0x7f8a100b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.289+0000 7f8a18311640 1 -- 172.21.15.16:0/1665653090 >> 172.21.15.16:0/1665653090 conn(0x7f8a1006d1a0 msgr2=0x7f8a10159590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.289+0000 7f8a18311640 1 -- 172.21.15.16:0/1665653090 shutdown_connections 2024-09-12T01:33:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.289+0000 7f8a18311640 1 -- 172.21.15.16:0/1665653090 wait complete. 2024-09-12T01:33:33.300 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-12T01:33:33.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:33:33.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.404+0000 7f0b29caf640 1 Processor -- start 2024-09-12T01:33:33.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.404+0000 7f0b29caf640 1 -- start start 2024-09-12T01:33:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.404+0000 7f0b29caf640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0b2406eda0 0x7f0b240af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.404+0000 7f0b29caf640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b2406cd10 0x7f0b240af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.404+0000 7f0b29caf640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b240cf150 con 0x7f0b2415e3f0 2024-09-12T01:33:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.404+0000 7f0b29caf640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f0b240cf430 con 0x7f0b2406eda0 2024-09-12T01:33:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.404+0000 7f0b29caf640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0b240ae880 con 0x7f0b2406cd10 2024-09-12T01:33:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.405+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b2406cd10 0x7f0b240af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.405+0000 7f0b22ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0b2406eda0 0x7f0b240af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.405+0000 7f0b237fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0b2415e3f0 0x7f0b2406e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:33446/0 (socket says 172.21.15.16:33446) 2024-09-12T01:33:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.405+0000 7f0b22ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0b2406eda0 0x7f0b240af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45392/0 (socket says 172.21.15.16:45392) 2024-09-12T01:33:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.405+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b2406cd10 0x7f0b240af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44276/0 (socket says 172.21.15.16:44276) 2024-09-12T01:33:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.405+0000 7f0b22ffd640 1 -- 172.21.15.16:0/3654100508 learned_addr learned my addr 172.21.15.16:0/3654100508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.405+0000 7f0b22ffd640 1 -- 172.21.15.16:0/3654100508 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b2406cd10 msgr2=0x7f0b240af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.405+0000 7f0b22ffd640 1 --2- 172.21.15.16:0/3654100508 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b2406cd10 0x7f0b240af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.405+0000 7f0b22ffd640 1 -- 172.21.15.16:0/3654100508 >> v1:172.21.15.33:6789/0 conn(0x7f0b2415e3f0 legacy=0x7f0b2406e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.405+0000 7f0b22ffd640 1 -- 172.21.15.16:0/3654100508 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b240aeb60 con 0x7f0b2406eda0 2024-09-12T01:33:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.405+0000 7f0b227fc640 1 -- 172.21.15.16:0/3654100508 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2839986988 0 0) 0x7f0b240cf150 con 0x7f0b2415e3f0 2024-09-12T01:33:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.405+0000 7f0b23fff640 1 --2- 172.21.15.16:0/3654100508 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b2406cd10 0x7f0b240af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.406+0000 7f0b22ffd640 1 --2- 172.21.15.16:0/3654100508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b2406eda0 0x7f0b240af360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009870 tx=0x7f0b1802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93f0625545d09082 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.406+0000 7f0b227fc640 1 -- 172.21.15.16:0/3654100508 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18037070 con 0x7f0b2406eda0 2024-09-12T01:33:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.406+0000 7f0b227fc640 1 -- 172.21.15.16:0/3654100508 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b18032440 con 0x7f0b2406eda0 2024-09-12T01:33:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.406+0000 7f0b227fc640 1 -- 172.21.15.16:0/3654100508 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18032810 con 0x7f0b2406eda0 2024-09-12T01:33:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.407+0000 7f0b29caf640 1 -- 172.21.15.16:0/3654100508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b2406eda0 msgr2=0x7f0b240af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.407+0000 7f0b29caf640 1 --2- 172.21.15.16:0/3654100508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b2406eda0 0x7f0b240af360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009870 tx=0x7f0b1802a270 comp rx=0 tx=0).stop 2024-09-12T01:33:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.407+0000 7f0b29caf640 1 -- 172.21.15.16:0/3654100508 shutdown_connections 2024-09-12T01:33:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.407+0000 7f0b29caf640 1 --2- 172.21.15.16:0/3654100508 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b2406cd10 0x7f0b240af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.407+0000 7f0b29caf640 1 --2- 172.21.15.16:0/3654100508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b2406eda0 0x7f0b240af360 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.407+0000 7f0b29caf640 1 -- 172.21.15.16:0/3654100508 >> 172.21.15.16:0/3654100508 conn(0x7f0b240ce2d0 msgr2=0x7f0b240723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.407+0000 7f0b29caf640 1 -- 172.21.15.16:0/3654100508 shutdown_connections 2024-09-12T01:33:33.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.407+0000 7f0b29caf640 1 -- 172.21.15.16:0/3654100508 wait complete. 2024-09-12T01:33:33.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.408+0000 7f0b29caf640 1 Processor -- start 2024-09-12T01:33:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.408+0000 7f0b29caf640 1 -- start start 2024-09-12T01:33:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b29caf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b2406cd10 0x7f0b24157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b29caf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b2406eda0 0x7f0b2415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b29caf640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b2415e3f0 0x7f0b2415ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b29caf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b240d0010 con 0x7f0b2406cd10 2024-09-12T01:33:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b29caf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b240b3980 con 0x7f0b2406eda0 2024-09-12T01:33:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b29caf640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0b240d0190 con 0x7f0b2415e3f0 2024-09-12T01:33:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b2406cd10 0x7f0b24157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b2406cd10 0x7f0b24157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42072/0 (socket says 172.21.15.16:42072) 2024-09-12T01:33:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b237fe640 1 -- 172.21.15.16:0/888110383 learned_addr learned my addr 172.21.15.16:0/888110383 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b22ffd640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b2406eda0 0x7f0b2415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b23fff640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b2415e3f0 0x7f0b2415ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b237fe640 1 -- 172.21.15.16:0/888110383 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b2415e3f0 msgr2=0x7f0b2415ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b237fe640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b2415e3f0 0x7f0b2415ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b237fe640 1 -- 172.21.15.16:0/888110383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b2406eda0 msgr2=0x7f0b2415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b237fe640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b2406eda0 0x7f0b2415fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.409+0000 7f0b237fe640 1 -- 172.21.15.16:0/888110383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b24157600 con 0x7f0b2406cd10 2024-09-12T01:33:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.410+0000 7f0b23fff640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b2415e3f0 0x7f0b2415ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.410+0000 7f0b22ffd640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b2406eda0 0x7f0b2415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.410+0000 7f0b237fe640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b2406cd10 0x7f0b24157cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400c430 tx=0x7f0b1400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.410+0000 7f0b0bfff640 1 -- 172.21.15.16:0/888110383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b140074e0 con 0x7f0b2406cd10 2024-09-12T01:33:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.410+0000 7f0b0bfff640 1 -- 172.21.15.16:0/888110383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b140047e0 con 0x7f0b2406cd10 2024-09-12T01:33:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.410+0000 7f0b29caf640 1 -- 172.21.15.16:0/888110383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b241578e0 con 0x7f0b2406cd10 2024-09-12T01:33:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.410+0000 7f0b0bfff640 1 -- 172.21.15.16:0/888110383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14004bd0 con 0x7f0b2406cd10 2024-09-12T01:33:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.410+0000 7f0b29caf640 1 -- 172.21.15.16:0/888110383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b24159bf0 con 0x7f0b2406cd10 2024-09-12T01:33:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.411+0000 7f0b29caf640 1 -- 172.21.15.16:0/888110383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b2415e7d0 con 0x7f0b2406cd10 2024-09-12T01:33:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.412+0000 7f0b0bfff640 1 -- 172.21.15.16:0/888110383 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b14011070 con 0x7f0b2406cd10 2024-09-12T01:33:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.412+0000 7f0b0bfff640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0b1006be30 0x7f0b1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.412+0000 7f0b0bfff640 1 -- 172.21.15.16:0/888110383 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0b1408ae30 con 0x7f0b2406cd10 2024-09-12T01:33:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.412+0000 7f0b22ffd640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0b1006be30 0x7f0b1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.412+0000 7f0b22ffd640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0b1006be30 0x7f0b1006e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b18034000 tx=0x7f0b1802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.416+0000 7f0b0bfff640 1 -- 172.21.15.16:0/888110383 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0b140594b0 con 0x7f0b2406cd10 2024-09-12T01:33:33.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.594+0000 7f0b29caf640 1 -- 172.21.15.16:0/888110383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b240cf150 con 0x7f0b2406cd10 2024-09-12T01:33:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.595+0000 7f0b0bfff640 1 -- 172.21.15.16:0/888110383 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18574 (secure 0 0 0) 0x7f0b14058db0 con 0x7f0b2406cd10 2024-09-12T01:33:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:33.595 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:33:18.213622+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:33:09.364639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:33:10.478237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:33:14.204624+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:33:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.597+0000 7f0b29caf640 1 -- 172.21.15.16:0/888110383 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0b1006be30 msgr2=0x7f0b1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.597+0000 7f0b29caf640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0b1006be30 0x7f0b1006e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b18034000 tx=0x7f0b1802e000 comp rx=0 tx=0).stop 2024-09-12T01:33:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.597+0000 7f0b29caf640 1 -- 172.21.15.16:0/888110383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b2406cd10 msgr2=0x7f0b24157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.597+0000 7f0b29caf640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b2406cd10 0x7f0b24157cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400c430 tx=0x7f0b1400c8f0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.597+0000 7f0b29caf640 1 -- 172.21.15.16:0/888110383 shutdown_connections 2024-09-12T01:33:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.597+0000 7f0b29caf640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0b1006be30 0x7f0b1006e2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.597+0000 7f0b29caf640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0b2415e3f0 0x7f0b2415ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.597+0000 7f0b29caf640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b2406eda0 0x7f0b2415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.597+0000 7f0b29caf640 1 --2- 172.21.15.16:0/888110383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b2406cd10 0x7f0b24157cd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.597+0000 7f0b29caf640 1 -- 172.21.15.16:0/888110383 >> 172.21.15.16:0/888110383 conn(0x7f0b240ce2d0 msgr2=0x7f0b2406e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.597+0000 7f0b29caf640 1 -- 172.21.15.16:0/888110383 shutdown_connections 2024-09-12T01:33:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.597+0000 7f0b29caf640 1 -- 172.21.15.16:0/888110383 wait complete. 2024-09-12T01:33:33.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-12T01:33:33.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-12T01:33:33.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-12T01:33:33.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-12T01:33:33.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-12T01:33:33.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-12T01:33:33.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-12T01:33:33.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-12T01:33:33.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.787+0000 7fef02282640 1 Processor -- start 2024-09-12T01:33:33.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.787+0000 7fef02282640 1 -- start start 2024-09-12T01:33:33.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.787+0000 7f84cca30640 1 Processor -- start 2024-09-12T01:33:33.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.788+0000 7fef02282640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feefc0cd360 0x7feefc159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.789+0000 7fef02282640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feefc15a7f0 con 0x7feefc15a360 2024-09-12T01:33:33.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.787+0000 7f84cca30640 1 -- start start 2024-09-12T01:33:33.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.789+0000 7f84cca30640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f84c80cd360 0x7f84c8159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.789+0000 7f84cca30640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84c815a7f0 con 0x7f84c80ccf80 2024-09-12T01:33:33.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.789+0000 7f84cca30640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84c815e260 con 0x7f84c815a360 2024-09-12T01:33:33.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.789+0000 7f84cca30640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f84c815ede0 con 0x7f84c80cd360 2024-09-12T01:33:33.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.789+0000 7feefb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feefc0cd360 0x7feefc159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.789+0000 7feefb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feefc0cd360 0x7feefc159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42082/0 (socket says 172.21.15.16:42082) 2024-09-12T01:33:33.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.789+0000 7f84c659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f84c80ccf80 0x7f84c80af090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53612/0 (socket says 172.21.15.16:53612) 2024-09-12T01:33:33.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.789+0000 7f84c659c640 1 -- 172.21.15.16:0/1404963176 learned_addr learned my addr 172.21.15.16:0/1404963176 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:33.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.789+0000 7feefb7fe640 1 -- 172.21.15.16:0/2317564946 learned_addr learned my addr 172.21.15.16:0/2317564946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.789+0000 7fef02282640 1 -- 172.21.15.16:0/2317564946 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feefc15e260 con 0x7feefc0ccf80 2024-09-12T01:33:33.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.789+0000 7fef02282640 1 -- 172.21.15.16:0/2317564946 --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7feefc15ede0 con 0x7feefc0cd360 2024-09-12T01:33:33.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.789+0000 7f84c5d9b640 1 --2- 172.21.15.16:0/1404963176 >> v2:172.21.15.33:3300/0 conn(0x7f84c80cd360 0x7f84c8159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.790+0000 7f84c5d9b640 1 -- 172.21.15.16:0/1404963176 >> v1:172.21.15.33:6789/0 conn(0x7f84c815a360 legacy=0x7f84c80ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.790+0000 7f84c5d9b640 1 -- 172.21.15.16:0/1404963176 >> v1:172.21.15.16:6789/0 conn(0x7f84c80ccf80 legacy=0x7f84c80af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.790+0000 7f84c5d9b640 1 -- 172.21.15.16:0/1404963176 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c815e540 con 0x7f84c80cd360 2024-09-12T01:33:33.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.790+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3693717182 0 0) 0x7feefc15e260 con 0x7feefc0ccf80 2024-09-12T01:33:33.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.790+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feee4003520 con 0x7feefc0ccf80 2024-09-12T01:33:33.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.790+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 <== mon.1 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2278095175 0 0) 0x7feee4003520 con 0x7feefc0ccf80 2024-09-12T01:33:33.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.790+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 >> v2:172.21.15.16:3300/0 conn(0x7feefc0cd360 msgr2=0x7feefc159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.790+0000 7feefaffd640 1 --2- 172.21.15.16:0/2317564946 >> v2:172.21.15.16:3300/0 conn(0x7feefc0cd360 0x7feefc159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.790+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 >> v1:172.21.15.16:6789/0 conn(0x7feefc15a360 legacy=0x7feefc0ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.791+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc15e540 con 0x7feefc0ccf80 2024-09-12T01:33:33.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.791+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 <== mon.1 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7feef40019f0 con 0x7feefc0ccf80 2024-09-12T01:33:33.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.791+0000 7f84c5d9b640 1 --2- 172.21.15.16:0/1404963176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c80cd360 0x7f84c8159e20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0097a0 tx=0x7f84bc02e740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84acfedf60701d18 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.791+0000 7f3a2af28640 1 Processor -- start 2024-09-12T01:33:33.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.791+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 >> v1:172.21.15.33:6789/0 conn(0x7feefc0ccf80 legacy=0x7feefc0af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.791+0000 7feefaffd640 1 --2- 172.21.15.16:0/2317564946 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feee4003a20 0x7feee4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.791+0000 7feefaffd640 1 --2- 172.21.15.16:0/2317564946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee4004800 0x7feee4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.791+0000 7f84c559a640 1 -- 172.21.15.16:0/1404963176 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc03a070 con 0x7f84c80cd360 2024-09-12T01:33:33.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.791+0000 7f84c559a640 1 -- 172.21.15.16:0/1404963176 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84bc033710 con 0x7f84c80cd360 2024-09-12T01:33:33.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.791+0000 7f6cc359e640 1 Processor -- start 2024-09-12T01:33:33.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.792+0000 7f6cc359e640 1 -- start start 2024-09-12T01:33:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.791+0000 7feefbfff640 1 --2- 172.21.15.16:0/2317564946 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feee4003a20 0x7feee4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.792+0000 7feefaffd640 1 --2- 172.21.15.16:0/2317564946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee40051c0 0x7feee400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.792+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feefc15ede0 con 0x7feee4004800 2024-09-12T01:33:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.792+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feee400c010 con 0x7feee40051c0 2024-09-12T01:33:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.792+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7feee400c210 con 0x7feee4003a20 2024-09-12T01:33:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.792+0000 7f84c559a640 1 -- 172.21.15.16:0/1404963176 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc033ae0 con 0x7f84c80cd360 2024-09-12T01:33:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.792+0000 7fabe59c5640 1 Processor -- start 2024-09-12T01:33:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.792+0000 7feefb7fe640 1 --2- 172.21.15.16:0/2317564946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee4004800 0x7feee4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.791+0000 7f3a2af28640 1 -- start start 2024-09-12T01:33:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.792+0000 7f84cca30640 1 -- 172.21.15.16:0/1404963176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c80cd360 msgr2=0x7f84c8159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.792+0000 7f84cca30640 1 --2- 172.21.15.16:0/1404963176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c80cd360 0x7f84c8159e20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0097a0 tx=0x7f84bc02e740 comp rx=0 tx=0).stop 2024-09-12T01:33:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.792+0000 7fabe59c5640 1 -- start start 2024-09-12T01:33:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.792+0000 7feefb7fe640 1 -- 172.21.15.16:0/2317564946 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feee4003a20 msgr2=0x7feee4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7feefb7fe640 1 --2- 172.21.15.16:0/2317564946 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feee4003a20 0x7feee4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7feefb7fe640 1 -- 172.21.15.16:0/2317564946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee40051c0 msgr2=0x7feee400b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T01:33:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7feefb7fe640 1 --2- 172.21.15.16:0/2317564946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee40051c0 0x7feee400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7feefb7fe640 1 -- 172.21.15.16:0/2317564946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee4003520 con 0x7feee4004800 2024-09-12T01:33:33.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7fef0081f640 1 -- 172.21.15.16:0/2317564946 reap_dead start 2024-09-12T01:33:33.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f84cca30640 1 -- 172.21.15.16:0/1404963176 shutdown_connections 2024-09-12T01:33:33.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f84cca30640 1 --2- 172.21.15.16:0/1404963176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c80cd360 0x7f84c8159e20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f84cca30640 1 -- 172.21.15.16:0/1404963176 >> 172.21.15.16:0/1404963176 conn(0x7f84c80b1af0 msgr2=0x7f84c80b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f3a2af28640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a240c2f70 0x7f3a240cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f3a2af28640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3a24163980 0x7f3a240c3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f3a2af28640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3a240cef70 0x7f3a240cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f3a2af28640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3a240cbf90 con 0x7f3a240c2f70 2024-09-12T01:33:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f3a2af28640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f3a240c3890 con 0x7f3a240cef70 2024-09-12T01:33:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f3a2af28640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f3a240cf8a0 con 0x7f3a24163980 2024-09-12T01:33:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f84cca30640 1 -- 172.21.15.16:0/1404963176 shutdown_connections 2024-09-12T01:33:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f3a23fff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3a24163980 0x7f3a240c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f3a23fff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3a24163980 0x7f3a240c3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44306/0 (socket says 172.21.15.16:44306) 2024-09-12T01:33:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f3a23fff640 1 -- 172.21.15.16:0/3304508221 learned_addr learned my addr 172.21.15.16:0/3304508221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f3a28cc4640 1 --2- 172.21.15.16:0/3304508221 >> v2:172.21.15.16:3300/0 conn(0x7f3a240c2f70 0x7f3a240cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f3a294c5640 1 --2- 172.21.15.16:0/3304508221 >> v2:172.21.15.33:3300/0 conn(0x7f3a240cef70 0x7f3a240cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7fabe59c5640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fabd80373a0 0x7fabd80088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7fabe59c5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fabd8034b40 0x7fabd8053c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7fabe59c5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fabd8054140 0x7fabd803aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7f3a23fff640 1 -- 172.21.15.16:0/3304508221 >> v2:172.21.15.33:3300/0 conn(0x7f3a240cef70 msgr2=0x7f3a240cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7f3a23fff640 1 --2- 172.21.15.16:0/3304508221 >> v2:172.21.15.33:3300/0 conn(0x7f3a240cef70 0x7f3a240cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7f3a23fff640 1 -- 172.21.15.16:0/3304508221 >> v2:172.21.15.16:3300/0 conn(0x7f3a240c2f70 msgr2=0x7f3a240cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7f3a23fff640 1 --2- 172.21.15.16:0/3304508221 >> v2:172.21.15.16:3300/0 conn(0x7f3a240c2f70 0x7f3a240cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7f3a23fff640 1 -- 172.21.15.16:0/3304508221 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a240cb940 con 0x7f3a24163980 2024-09-12T01:33:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7f1b4b623640 1 Processor -- start 2024-09-12T01:33:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7f6cc259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6cc415e3f0 0x7f6cc40d43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53614/0 (socket says 172.21.15.16:53614) 2024-09-12T01:33:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7f6cc259c640 1 -- 172.21.15.16:0/1612602548 learned_addr learned my addr 172.21.15.16:0/1612602548 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7f6cc359e640 1 --2- 172.21.15.16:0/1612602548 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cc40d0770 0x7f6cc41691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7feefb7fe640 1 --2- 172.21.15.16:0/2317564946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee4004800 0x7feee4004c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feef00097a0 tx=0x7feee4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b967cbb3b6d1dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef00097e0 con 0x7feee4004800 2024-09-12T01:33:33.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef0032410 con 0x7feee4004800 2024-09-12T01:33:33.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7feefaffd640 1 -- 172.21.15.16:0/2317564946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef00327e0 con 0x7feee4004800 2024-09-12T01:33:33.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.793+0000 7f84cca30640 1 -- 172.21.15.16:0/1404963176 wait complete. 2024-09-12T01:33:33.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7f3a23fff640 1 --2- 172.21.15.16:0/3304508221 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3a24163980 0x7f3a240c3350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a240700c0 tx=0x7f3a1002a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a750cebbb7286d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7f6cc2d9d640 1 --2- 172.21.15.16:0/1612602548 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cc40d0770 0x7f6cc41691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7f6cc359e640 1 -- 172.21.15.16:0/1612602548 --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cc40b3ed0 con 0x7f6cc415e3f0 2024-09-12T01:33:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7f84cca30640 1 Processor -- start 2024-09-12T01:33:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1485664057 0 0) 0x7f6cb4001110 con 0x7f6cc415e3f0 2024-09-12T01:33:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7fef02282640 1 -- 172.21.15.16:0/2317564946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee4004800 msgr2=0x7feee4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7fef02282640 1 --2- 172.21.15.16:0/2317564946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee4004800 0x7feee4004c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feef00097a0 tx=0x7feee4005830 comp rx=0 tx=0).stop 2024-09-12T01:33:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7f6cc359e640 1 -- 172.21.15.16:0/1612602548 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cc40d2bd0 con 0x7f6cc40d4ac0 2024-09-12T01:33:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7f6cc359e640 1 -- 172.21.15.16:0/1612602548 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc40d2eb0 con 0x7f6cc40d0770 2024-09-12T01:33:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f3a237fe640 1 -- 172.21.15.16:0/3304508221 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10037070 con 0x7f3a24163980 2024-09-12T01:33:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f3a237fe640 1 -- 172.21.15.16:0/3304508221 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a10032440 con 0x7f3a24163980 2024-09-12T01:33:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7f84cca30640 1 -- start start 2024-09-12T01:33:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f3a237fe640 1 -- 172.21.15.16:0/3304508221 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10037070 con 0x7f3a24163980 2024-09-12T01:33:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f82dacf0640 1 Processor -- start 2024-09-12T01:33:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7f1b4b623640 1 -- start start 2024-09-12T01:33:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f1b4b623640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b440d2e00 0x7f1b440d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7fabe59c5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fabd803a360 con 0x7fabd8054140 2024-09-12T01:33:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7fabe59c5640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fabd800ae40 con 0x7fabd8034b40 2024-09-12T01:33:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7fabe59c5640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fabd800b120 con 0x7fabd80373a0 2024-09-12T01:33:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7fabdffff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fabd8034b40 0x7fabd8053c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7fabe49c3640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fabd80373a0 0x7fabd80088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7fabdffff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fabd8034b40 0x7fabd8053c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45442/0 (socket says 172.21.15.16:45442) 2024-09-12T01:33:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7fabdffff640 1 -- 172.21.15.16:0/4246070108 learned_addr learned my addr 172.21.15.16:0/4246070108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.794+0000 7fabe51c4640 1 --2- 172.21.15.16:0/4246070108 >> v2:172.21.15.16:3300/0 conn(0x7fabd8054140 0x7fabd803aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7fabdffff640 1 -- 172.21.15.16:0/4246070108 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fabd80373a0 msgr2=0x7fabd80088c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7fabdffff640 1 --2- 172.21.15.16:0/4246070108 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fabd80373a0 0x7fabd80088c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7fabdffff640 1 -- 172.21.15.16:0/4246070108 >> v2:172.21.15.16:3300/0 conn(0x7fabd8054140 msgr2=0x7fabd803aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7fabdffff640 1 --2- 172.21.15.16:0/4246070108 >> v2:172.21.15.16:3300/0 conn(0x7fabd8054140 0x7fabd803aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.795+0000 7fabdffff640 1 -- 172.21.15.16:0/4246070108 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd803a5b0 con 0x7fabd8034b40 2024-09-12T01:33:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7fabe51c4640 1 --2- 172.21.15.16:0/4246070108 >> v2:172.21.15.16:3300/0 conn(0x7fabd8054140 0x7fabd803aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f1b4b623640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b440d31e0 0x7f1b440d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f1b4b623640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b440d5220 con 0x7f1b440c7670 2024-09-12T01:33:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f1b4b623640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1b440d4ff0 con 0x7f1b440d2e00 2024-09-12T01:33:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f1b4b623640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f1b440d54e0 con 0x7f1b440d31e0 2024-09-12T01:33:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f82dacf0640 1 -- start start 2024-09-12T01:33:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7fef02282640 1 -- 172.21.15.16:0/2317564946 shutdown_connections 2024-09-12T01:33:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7fef02282640 1 --2- 172.21.15.16:0/2317564946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee4004800 0x7feee4004c80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7fef02282640 1 -- 172.21.15.16:0/2317564946 >> 172.21.15.16:0/2317564946 conn(0x7feefc0b1af0 msgr2=0x7feefc0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f1b49bc0640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1b440c7670 0x7f1b440cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:33496/0 (socket says 172.21.15.16:33496) 2024-09-12T01:33:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f1b49bc0640 1 -- 172.21.15.16:0/4294678582 learned_addr learned my addr 172.21.15.16:0/4294678582 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f1b48bbe640 1 --2- 172.21.15.16:0/4294678582 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b440d31e0 0x7f1b440d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7fabdffff640 1 --2- 172.21.15.16:0/4246070108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd8034b40 0x7fabd8053c00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fabd800fc80 tx=0x7fabd402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd3adee693181731 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f84cca30640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84c80ccf80 0x7f84c80adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f84cca30640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c80cd360 0x7f84c80ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84cca30640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c815a360 0x7f84c815ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84cca30640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84c80af7a0 con 0x7f84c815a360 2024-09-12T01:33:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84cca30640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84c815eb00 con 0x7f84c80cd360 2024-09-12T01:33:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84cca30640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f84c815def0 con 0x7f84c80ccf80 2024-09-12T01:33:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84c5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c80cd360 0x7f84c80ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cac003520 con 0x7f6cc415e3f0 2024-09-12T01:33:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3080337401 0 0) 0x7f6cc40d2bd0 con 0x7f6cc40d4ac0 2024-09-12T01:33:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cb4001110 con 0x7f6cc40d4ac0 2024-09-12T01:33:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7fabdf7fe640 1 -- 172.21.15.16:0/4246070108 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4037070 con 0x7fabd8034b40 2024-09-12T01:33:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7fabdf7fe640 1 -- 172.21.15.16:0/4246070108 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabd4032440 con 0x7fabd8034b40 2024-09-12T01:33:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.796+0000 7f1b493bf640 1 --2- 172.21.15.16:0/4294678582 >> v2:172.21.15.16:3300/0 conn(0x7f1b440d2e00 0x7f1b440d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1007769741 0 0) 0x7f1b440d5220 con 0x7f1b440c7670 2024-09-12T01:33:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b30003520 con 0x7f1b440c7670 2024-09-12T01:33:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84c659c640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84c80ccf80 0x7f84c80adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84c5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c80cd360 0x7f84c80ae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45448/0 (socket says 172.21.15.16:45448) 2024-09-12T01:33:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84c5d9b640 1 -- 172.21.15.16:0/1637298507 learned_addr learned my addr 172.21.15.16:0/1637298507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7fef02282640 1 -- 172.21.15.16:0/2317564946 shutdown_connections 2024-09-12T01:33:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f3a2af28640 1 -- 172.21.15.16:0/3304508221 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3a24163980 msgr2=0x7f3a240c3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f3a2af28640 1 --2- 172.21.15.16:0/3304508221 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3a24163980 0x7f3a240c3350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a240700c0 tx=0x7f3a1002a1d0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7fef02282640 1 -- 172.21.15.16:0/2317564946 wait complete. 2024-09-12T01:33:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f3a2af28640 1 -- 172.21.15.16:0/3304508221 shutdown_connections 2024-09-12T01:33:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f3a2af28640 1 --2- 172.21.15.16:0/3304508221 >> v2:172.21.15.33:3300/0 conn(0x7f3a240cef70 0x7f3a240cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f3a2af28640 1 --2- 172.21.15.16:0/3304508221 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3a24163980 0x7f3a240c3350 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f3a2af28640 1 --2- 172.21.15.16:0/3304508221 >> v2:172.21.15.16:3300/0 conn(0x7f3a240c2f70 0x7f3a240cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84c6d9d640 1 --2- 172.21.15.16:0/1637298507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c815a360 0x7f84c815ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f3a2af28640 1 -- 172.21.15.16:0/3304508221 >> 172.21.15.16:0/3304508221 conn(0x7f3a240cffa0 msgr2=0x7f3a240d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7fabdf7fe640 1 -- 172.21.15.16:0/4246070108 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4032810 con 0x7fabd8034b40 2024-09-12T01:33:33.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 <== mon.1 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2138388928 0 0) 0x7f1b30003520 con 0x7f1b440c7670 2024-09-12T01:33:33.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b440d31e0 msgr2=0x7f1b440d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f1b3bfff640 1 --2- 172.21.15.16:0/4294678582 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b440d31e0 0x7f1b440d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 >> v2:172.21.15.16:3300/0 conn(0x7f1b440d2e00 msgr2=0x7f1b440d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f1b3bfff640 1 --2- 172.21.15.16:0/4294678582 >> v2:172.21.15.16:3300/0 conn(0x7f1b440d2e00 0x7f1b440d2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b440d0d80 con 0x7f1b440c7670 2024-09-12T01:33:33.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 <== mon.1 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f1b3c0019f0 con 0x7f1b440c7670 2024-09-12T01:33:33.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 >> v1:172.21.15.33:6789/0 conn(0x7f1b440c7670 legacy=0x7f1b440cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84c5d9b640 1 -- 172.21.15.16:0/1637298507 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84c80ccf80 msgr2=0x7f84c80adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84c5d9b640 1 --2- 172.21.15.16:0/1637298507 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84c80ccf80 0x7f84c80adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84c5d9b640 1 -- 172.21.15.16:0/1637298507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c815a360 msgr2=0x7f84c815ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84c5d9b640 1 --2- 172.21.15.16:0/1637298507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c815a360 0x7f84c815ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f84c5d9b640 1 -- 172.21.15.16:0/1637298507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c815d830 con 0x7f84c80cd360 2024-09-12T01:33:33.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7fabe59c5640 1 -- 172.21.15.16:0/4246070108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd8034b40 msgr2=0x7fabd8053c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7fabe59c5640 1 --2- 172.21.15.16:0/4246070108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd8034b40 0x7fabd8053c00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fabd800fc80 tx=0x7fabd402a130 comp rx=0 tx=0).stop 2024-09-12T01:33:33.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b3bfff640 1 --2- 172.21.15.16:0/4294678582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b30003a20 0x7f1b30003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b3bfff640 1 --2- 172.21.15.16:0/4294678582 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b440d31e0 0x7f1b30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7fabe59c5640 1 -- 172.21.15.16:0/4246070108 shutdown_connections 2024-09-12T01:33:33.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7fabe59c5640 1 --2- 172.21.15.16:0/4246070108 >> v2:172.21.15.16:3300/0 conn(0x7fabd8054140 0x7fabd803aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7fabe59c5640 1 --2- 172.21.15.16:0/4246070108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd8034b40 0x7fabd8053c00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f82dacf0640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f82d4073ac0 0x7f82d40aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f82dacf0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f82d415e530 0x7f82d406db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f82dacf0640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82d40b2f70 con 0x7f82d415e150 2024-09-12T01:33:33.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f82dacf0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f82d4073020 con 0x7f82d415e530 2024-09-12T01:33:33.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7fabe59c5640 1 --2- 172.21.15.16:0/4246070108 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fabd80373a0 0x7fabd80088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7fabe59c5640 1 -- 172.21.15.16:0/4246070108 >> 172.21.15.16:0/4246070108 conn(0x7fabd801ea40 msgr2=0x7fabd80556b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f82dacf0640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f82d40b3250 con 0x7f82d4073ac0 2024-09-12T01:33:33.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b3bfff640 1 --2- 172.21.15.16:0/4294678582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b30004d40 0x7f1b300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 404375635 0 0) 0x7f6cac003520 con 0x7f6cc415e3f0 2024-09-12T01:33:33.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cc40d0770 msgr2=0x7f6cc41691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f6cc159a640 1 --2- 172.21.15.16:0/1612602548 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cc40d0770 0x7f6cc41691d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b440d54e0 con 0x7f1b30004d40 2024-09-12T01:33:33.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3000bad0 con 0x7f1b30003a20 2024-09-12T01:33:33.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3000bcd0 con 0x7f1b440d31e0 2024-09-12T01:33:33.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b493bf640 1 --2- 172.21.15.16:0/4294678582 >> v2:172.21.15.16:3300/0 conn(0x7f1b440d2e00 0x7f1b440d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:33.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f82d94ed640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f82d4073ac0 0x7f82d40aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f84c5d9b640 1 --2- 172.21.15.16:0/1637298507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c80cd360 0x7f84c80ae4a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84bc02ed30 tx=0x7f84bc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f82d94ed640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f82d4073ac0 0x7f82d40aceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45456/0 (socket says 172.21.15.16:45456) 2024-09-12T01:33:33.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f82d94ed640 1 -- 172.21.15.16:0/2654890647 learned_addr learned my addr 172.21.15.16:0/2654890647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b49bc0640 1 --2- 172.21.15.16:0/4294678582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b30004d40 0x7f1b300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b493bf640 1 --2- 172.21.15.16:0/4294678582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b30003a20 0x7f1b30003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 >> v1:172.21.15.33:6789/0 conn(0x7f6cc40d4ac0 legacy=0x7f6cc40d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc40d3190 con 0x7f6cc415e3f0 2024-09-12T01:33:33.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f6cb4001840 con 0x7f6cc415e3f0 2024-09-12T01:33:33.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f3a2af28640 1 -- 172.21.15.16:0/3304508221 shutdown_connections 2024-09-12T01:33:33.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 >> v1:172.21.15.16:6789/0 conn(0x7f6cc415e3f0 legacy=0x7f6cc40d43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f6cc159a640 1 --2- 172.21.15.16:0/1612602548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac004bd0 0x7f6cac004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f6cc159a640 1 --2- 172.21.15.16:0/1612602548 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cac0037a0 0x7f6cac003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f6cc159a640 1 --2- 172.21.15.16:0/1612602548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cac004160 0x7f6cac0141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc40d2eb0 con 0x7f6cac004160 2024-09-12T01:33:33.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac014770 con 0x7f6cac004bd0 2024-09-12T01:33:33.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.797+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f6cac014970 con 0x7f6cac0037a0 2024-09-12T01:33:33.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7fa717b52640 1 Processor -- start 2024-09-12T01:33:33.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f6cc1d9b640 1 --2- 172.21.15.16:0/1612602548 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cac0037a0 0x7f6cac003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b49bc0640 1 -- 172.21.15.16:0/4294678582 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b440d31e0 msgr2=0x7f1b30004800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T01:33:33.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b49bc0640 1 --2- 172.21.15.16:0/4294678582 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b440d31e0 0x7f1b30004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b49bc0640 1 -- 172.21.15.16:0/4294678582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b30003a20 msgr2=0x7f1b30003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b49bc0640 1 --2- 172.21.15.16:0/4294678582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b30003a20 0x7f1b30003e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7f1b49bc0640 1 -- 172.21.15.16:0/4294678582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b30003520 con 0x7f1b30004d40 2024-09-12T01:33:33.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7fabe59c5640 1 -- 172.21.15.16:0/4246070108 shutdown_connections 2024-09-12T01:33:33.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7fabe59c5640 1 -- 172.21.15.16:0/4246070108 wait complete. 2024-09-12T01:33:33.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f84b6ffd640 1 -- 172.21.15.16:0/1637298507 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc03a070 con 0x7f84c80cd360 2024-09-12T01:33:33.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f84b6ffd640 1 -- 172.21.15.16:0/1637298507 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84bc013620 con 0x7f84c80cd360 2024-09-12T01:33:33.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f84b6ffd640 1 -- 172.21.15.16:0/1637298507 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc00f9e0 con 0x7f84c80cd360 2024-09-12T01:33:33.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f84cca30640 1 -- 172.21.15.16:0/1637298507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c80aead0 con 0x7f84c80cd360 2024-09-12T01:33:33.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f84cca30640 1 -- 172.21.15.16:0/1637298507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c80b41d0 con 0x7f84c80cd360 2024-09-12T01:33:33.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f3a2af28640 1 -- 172.21.15.16:0/3304508221 wait complete. 2024-09-12T01:33:33.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f6cc259c640 1 --2- 172.21.15.16:0/1612602548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac004bd0 0x7f6cac004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.798+0000 7fa717b52640 1 -- start start 2024-09-12T01:33:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4264653083 0 0) 0x7f82d40b2f70 con 0x7f82d415e150 2024-09-12T01:33:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82bc003520 con 0x7f82d415e150 2024-09-12T01:33:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 <== mon.1 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2377280870 0 0) 0x7f82bc003520 con 0x7f82d415e150 2024-09-12T01:33:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 >> v2:172.21.15.33:3300/0 conn(0x7f82d4073ac0 msgr2=0x7f82d40aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f82d8cec640 1 --2- 172.21.15.16:0/2654890647 >> v2:172.21.15.33:3300/0 conn(0x7f82d4073ac0 0x7f82d40aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 >> v2:172.21.15.16:3300/0 conn(0x7f82d415e530 msgr2=0x7f82d406db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f82d8cec640 1 --2- 172.21.15.16:0/2654890647 >> v2:172.21.15.16:3300/0 conn(0x7f82d415e530 0x7f82d406db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d40b3530 con 0x7f82d415e150 2024-09-12T01:33:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 <== mon.1 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f82c4001670 con 0x7f82d415e150 2024-09-12T01:33:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 >> v1:172.21.15.33:6789/0 conn(0x7f82d415e150 legacy=0x7f82d40733b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7fef02282640 1 Processor -- start 2024-09-12T01:33:33.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7f82d8cec640 1 --2- 172.21.15.16:0/2654890647 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f82bc0039b0 0x7f82bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7f82d8cec640 1 --2- 172.21.15.16:0/2654890647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82bc004800 0x7f82bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7f82d8cec640 1 --2- 172.21.15.16:0/2654890647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc0052e0 0x7f82bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4073020 con 0x7f82bc0052e0 2024-09-12T01:33:33.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82d40b3250 con 0x7f82bc004800 2024-09-12T01:33:33.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f82bc00c230 con 0x7f82bc0039b0 2024-09-12T01:33:33.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7f1b49bc0640 1 --2- 172.21.15.16:0/4294678582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b30004d40 0x7f1b300051c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00a7d0 tx=0x7f1b30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb89f3ae4db034f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7fef02282640 1 -- start start 2024-09-12T01:33:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f6cc1d9b640 1 -- 172.21.15.16:0/1612602548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac004bd0 msgr2=0x7f6cac004fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f6cc1d9b640 1 --2- 172.21.15.16:0/1612602548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac004bd0 0x7f6cac004fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7fabe59c5640 1 Processor -- start 2024-09-12T01:33:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f6cc1d9b640 1 -- 172.21.15.16:0/1612602548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cac004160 msgr2=0x7f6cac0141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f6cc1d9b640 1 --2- 172.21.15.16:0/1612602548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cac004160 0x7f6cac0141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.799+0000 7f6cc1d9b640 1 -- 172.21.15.16:0/1612602548 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc40d3190 con 0x7f6cac0037a0 2024-09-12T01:33:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7fabe59c5640 1 -- start start 2024-09-12T01:33:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7f82d9cee640 1 --2- 172.21.15.16:0/2654890647 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f82bc0039b0 0x7f82bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7f6cc1d9b640 1 --2- 172.21.15.16:0/1612602548 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cac0037a0 0x7f6cac003c20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8001910 tx=0x7f6cac005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f54793fe076f8f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7fa717b52640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708057560 0x7fa708037c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7fa717b52640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7080567a0 con 0x7fa7080e4b40 2024-09-12T01:33:33.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7fa717b52640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa708056a80 con 0x7fa708053790 2024-09-12T01:33:33.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7fa717b52640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa7080579f0 con 0x7fa708057560 2024-09-12T01:33:33.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7fa7150ed640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa708053790 0x7fa708005df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:33522/0 (socket says 172.21.15.16:33522) 2024-09-12T01:33:33.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7fa7150ed640 1 -- 172.21.15.16:0/57381320 learned_addr learned my addr 172.21.15.16:0/57381320 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:33.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7f3a2af28640 1 Processor -- start 2024-09-12T01:33:33.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8037070 con 0x7f6cac0037a0 2024-09-12T01:33:33.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb802ac90 con 0x7f6cac0037a0 2024-09-12T01:33:33.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7f6cc159a640 1 -- 172.21.15.16:0/1612602548 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80014e0 con 0x7f6cac0037a0 2024-09-12T01:33:33.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 391355751 0 0) 0x7fa708056a80 con 0x7fa708053790 2024-09-12T01:33:33.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6fc003520 con 0x7fa708053790 2024-09-12T01:33:33.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7160ef640 1 --2- 172.21.15.16:0/57381320 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708057560 0x7fa708037c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 <== mon.1 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 895256132 0 0) 0x7fa6fc003520 con 0x7fa708053790 2024-09-12T01:33:33.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708057560 msgr2=0x7fa708037c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 --2- 172.21.15.16:0/57381320 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708057560 0x7fa708037c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 >> v1:172.21.15.16:6789/0 conn(0x7fa7080e4b40 legacy=0x7fa708053080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa708056d60 con 0x7fa708053790 2024-09-12T01:33:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 <== mon.1 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7fa710050800 con 0x7fa708053790 2024-09-12T01:33:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 >> v1:172.21.15.33:6789/0 conn(0x7fa708053790 legacy=0x7fa708005df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 --2- 172.21.15.16:0/57381320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6fc0039b0 0x7fa6fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 --2- 172.21.15.16:0/57381320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc0043c0 0x7fa6fc0048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 --2- 172.21.15.16:0/57381320 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708057560 0x7fa6fc0050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc00b9c0 con 0x7fa6fc0043c0 2024-09-12T01:33:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc00bb40 con 0x7fa6fc0039b0 2024-09-12T01:33:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa6fc00bd40 con 0x7fa708057560 2024-09-12T01:33:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fa7160ef640 1 --2- 172.21.15.16:0/57381320 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708057560 0x7fa6fc0050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fa7158ee640 1 --2- 172.21.15.16:0/57381320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6fc0039b0 0x7fa6fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fa7150ed640 1 --2- 172.21.15.16:0/57381320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc0043c0 0x7fa6fc0048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fa7150ed640 1 -- 172.21.15.16:0/57381320 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708057560 msgr2=0x7fa6fc0050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fa7150ed640 1 --2- 172.21.15.16:0/57381320 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708057560 0x7fa6fc0050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fa7150ed640 1 -- 172.21.15.16:0/57381320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6fc0039b0 msgr2=0x7fa6fc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fa7150ed640 1 --2- 172.21.15.16:0/57381320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6fc0039b0 0x7fa6fc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fa7150ed640 1 -- 172.21.15.16:0/57381320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa708056d60 con 0x7fa6fc0043c0 2024-09-12T01:33:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fef02282640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee4004800 0x7feefc1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fef02282640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feefc0c9900 0x7feefc0c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fef02282640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc0c7100 0x7feefc0c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fef02282640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feefc15def0 con 0x7feee4004800 2024-09-12T01:33:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fef02282640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feefc15eb00 con 0x7feefc0c7100 2024-09-12T01:33:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7fef02282640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7feefc0b25e0 con 0x7feefc0c9900 2024-09-12T01:33:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7feefbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee4004800 0x7feefc1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7feefbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee4004800 0x7feefc1597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42158/0 (socket says 172.21.15.16:42158) 2024-09-12T01:33:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7feefbfff640 1 -- 172.21.15.16:0/2588579860 learned_addr learned my addr 172.21.15.16:0/2588579860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fabe59c5640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fabd80373a0 0x7fabd80330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fabe59c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd8054140 0x7fabd8007620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fabe59c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd80e5320 0x7fabd8054ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fabe59c5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabd8037780 con 0x7fabd80e5320 2024-09-12T01:33:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fabe59c5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fabd80345f0 con 0x7fabd8054140 2024-09-12T01:33:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fabe59c5640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fabd8037900 con 0x7fabd80373a0 2024-09-12T01:33:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7f82da4ef640 1 --2- 172.21.15.16:0/2654890647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc0052e0 0x7f82bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7f82d94ed640 1 --2- 172.21.15.16:0/2654890647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82bc004800 0x7f82bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00a3f0 con 0x7f1b30004d40 2024-09-12T01:33:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3c02f070 con 0x7f1b30004d40 2024-09-12T01:33:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.800+0000 7f1b3bfff640 1 -- 172.21.15.16:0/4294678582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c02bcd0 con 0x7f1b30004d40 2024-09-12T01:33:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7f1b4b623640 1 -- 172.21.15.16:0/4294678582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b30004d40 msgr2=0x7f1b300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7f1b4b623640 1 --2- 172.21.15.16:0/4294678582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b30004d40 0x7f1b300051c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00a7d0 tx=0x7f1b30005830 comp rx=0 tx=0).stop 2024-09-12T01:33:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7f1b49bc0640 1 -- 172.21.15.16:0/4294678582 reap_dead start 2024-09-12T01:33:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fef0081f640 1 --2- 172.21.15.16:0/2588579860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc0c7100 0x7feefc0c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7feefbfff640 1 -- 172.21.15.16:0/2588579860 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feefc0c9900 msgr2=0x7feefc0c9d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:33:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7feefbfff640 1 --2- 172.21.15.16:0/2588579860 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feefc0c9900 0x7feefc0c9d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7feefbfff640 1 -- 172.21.15.16:0/2588579860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc0c7100 msgr2=0x7feefc0c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7feefbfff640 1 --2- 172.21.15.16:0/2588579860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc0c7100 0x7feefc0c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7feefbfff640 1 -- 172.21.15.16:0/2588579860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc15af30 con 0x7feee4004800 2024-09-12T01:33:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7f3a2af28640 1 -- start start 2024-09-12T01:33:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7f6cc359e640 1 -- 172.21.15.16:0/1612602548 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cac0037a0 msgr2=0x7f6cac003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7f6cc359e640 1 --2- 172.21.15.16:0/1612602548 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cac0037a0 0x7f6cac003c20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8001910 tx=0x7f6cac005830 comp rx=0 tx=0).stop 2024-09-12T01:33:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fabdffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd8054140 0x7fabd8007620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7f6cc2d9d640 1 -- 172.21.15.16:0/1612602548 reap_dead start 2024-09-12T01:33:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7fabdffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd8054140 0x7fabd8007620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45514/0 (socket says 172.21.15.16:45514) 2024-09-12T01:33:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.801+0000 7f84cca30640 1 -- 172.21.15.16:0/1637298507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84c80b25e0 con 0x7f84c80cd360 2024-09-12T01:33:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7fabdffff640 1 -- 172.21.15.16:0/3712574633 learned_addr learned my addr 172.21.15.16:0/3712574633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7feefbfff640 1 --2- 172.21.15.16:0/2588579860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee4004800 0x7feefc1597f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feef4006730 tx=0x7feef400f530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7f3a2af28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a240c2f70 0x7f3a24150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7fabe49c3640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fabd80373a0 0x7fabd80330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f3a2af28640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3a240cef70 0x7f3a240c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f3a2af28640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a24163980 0x7f3a24151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f1b4b623640 1 -- 172.21.15.16:0/4294678582 shutdown_connections 2024-09-12T01:33:33.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f1b4b623640 1 -- 172.21.15.16:0/4294678582 >> 172.21.15.16:0/4294678582 conn(0x7f1b4409d7c0 msgr2=0x7f1b440d45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f84b6ffd640 1 -- 172.21.15.16:0/1637298507 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84bc045060 con 0x7f84c80cd360 2024-09-12T01:33:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7fabe51c4640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd80e5320 0x7fabd8054ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f3a2af28640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a240cfb80 con 0x7f3a240c2f70 2024-09-12T01:33:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f3a2af28640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a24162a20 con 0x7f3a24163980 2024-09-12T01:33:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f3a2af28640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f3a240cf420 con 0x7f3a240cef70 2024-09-12T01:33:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7fa7150ed640 1 --2- 172.21.15.16:0/57381320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc0043c0 0x7fa6fc0048a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa7100637b0 tx=0x7fa6fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316dac34e8d39c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f3a294c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a24163980 0x7f3a24151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f3a28cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a240c2f70 0x7f3a24150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f3a294c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a24163980 0x7f3a24151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45530/0 (socket says 172.21.15.16:45530) 2024-09-12T01:33:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f3a294c5640 1 -- 172.21.15.16:0/1483398464 learned_addr learned my addr 172.21.15.16:0/1483398464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7fabdffff640 1 -- 172.21.15.16:0/3712574633 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fabd80373a0 msgr2=0x7fabd80330a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7f82da4ef640 1 -- 172.21.15.16:0/2654890647 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f82bc0039b0 msgr2=0x7f82bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7f82da4ef640 1 --2- 172.21.15.16:0/2654890647 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f82bc0039b0 0x7f82bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7f82da4ef640 1 -- 172.21.15.16:0/2654890647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82bc004800 msgr2=0x7f82bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7f82da4ef640 1 --2- 172.21.15.16:0/2654890647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82bc004800 0x7f82bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.802+0000 7f82da4ef640 1 -- 172.21.15.16:0/2654890647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d40b3530 con 0x7f82bc0052e0 2024-09-12T01:33:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f82da4ef640 1 -- 172.21.15.16:0/2654890647 reap_dead start 2024-09-12T01:33:33.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f84b6ffd640 1 --2- 172.21.15.16:0/1637298507 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f84a806bf30 0x7f84a806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7fabdffff640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fabd80373a0 0x7fabd80330a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7fabdffff640 1 -- 172.21.15.16:0/3712574633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd80e5320 msgr2=0x7fabd8054ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7fabdffff640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd80e5320 0x7fabd8054ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7f84b6ffd640 1 -- 172.21.15.16:0/1637298507 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f84bc0aaa70 con 0x7f84c80cd360 2024-09-12T01:33:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f3a294c5640 1 -- 172.21.15.16:0/1483398464 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3a240cef70 msgr2=0x7f3a240c9020 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T01:33:33.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7f3a294c5640 1 --2- 172.21.15.16:0/1483398464 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3a240cef70 0x7f3a240c9020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7f3a294c5640 1 -- 172.21.15.16:0/1483398464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a240c2f70 msgr2=0x7f3a24150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7f3a294c5640 1 --2- 172.21.15.16:0/1483398464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a240c2f70 0x7f3a24150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7f3a294c5640 1 -- 172.21.15.16:0/1483398464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a24151cd0 con 0x7f3a24163980 2024-09-12T01:33:33.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f6cc359e640 1 -- 172.21.15.16:0/1612602548 shutdown_connections 2024-09-12T01:33:33.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7f84b6ffd640 1 --2- 172.21.15.16:0/1637298507 >> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] conn(0x7f84a80756a0 0x7f84a8077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7fabdffff640 1 -- 172.21.15.16:0/3712574633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd80e81f0 con 0x7fabd8054140 2024-09-12T01:33:33.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7fabe51c4640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd80e5320 0x7fabd8054ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:33.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7fabdffff640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd8054140 0x7fabd8007620 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fabd40098a0 tx=0x7fabd40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f6cc359e640 1 -- 172.21.15.16:0/1612602548 >> 172.21.15.16:0/1612602548 conn(0x7f6cc408ddd0 msgr2=0x7f6cc409d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7f84c659c640 1 --2- 172.21.15.16:0/1637298507 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f84a806bf30 0x7f84a806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7f84b6ffd640 1 -- 172.21.15.16:0/1637298507 --> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84bc043e50 con 0x7f84a80756a0 2024-09-12T01:33:33.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7f84b6ffd640 1 -- 172.21.15.16:0/1637298507 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84bc0af2a0 con 0x7f84c80cd360 2024-09-12T01:33:33.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7f6cc359e640 1 -- 172.21.15.16:0/1612602548 shutdown_connections 2024-09-12T01:33:33.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7f84c6d9d640 1 --2- 172.21.15.16:0/1637298507 >> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] conn(0x7f84a80756a0 0x7f84a8077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7f6cc359e640 1 -- 172.21.15.16:0/1612602548 wait complete. 2024-09-12T01:33:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7f82da4ef640 1 --2- 172.21.15.16:0/2654890647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc0052e0 0x7f82bc00b9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f82d415e9a0 tx=0x7f82bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dbaaec51fa3816 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc034020 con 0x7f82bc0052e0 2024-09-12T01:33:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82cc006e50 con 0x7f82bc0052e0 2024-09-12T01:33:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7f3a294c5640 1 --2- 172.21.15.16:0/1483398464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a24163980 0x7f3a24151350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c007e40 tx=0x7f3a1c00e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7f84c6d9d640 1 --2- 172.21.15.16:0/1637298507 >> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] conn(0x7f84a80756a0 0x7f84a8077ae0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71006dc30 con 0x7fa6fc0043c0 2024-09-12T01:33:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa71006baf0 con 0x7fa6fc0043c0 2024-09-12T01:33:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.804+0000 7fa7148ec640 1 -- 172.21.15.16:0/57381320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71006d070 con 0x7fa6fc0043c0 2024-09-12T01:33:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7f84c659c640 1 --2- 172.21.15.16:0/1637298507 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f84a806bf30 0x7f84a806e3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f84c00012e0 tx=0x7f84c0006290 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7fa717b52640 1 -- 172.21.15.16:0/57381320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc0043c0 msgr2=0x7fa6fc0048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7fa717b52640 1 --2- 172.21.15.16:0/57381320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc0043c0 0x7fa6fc0048a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa7100637b0 tx=0x7fa6fc005830 comp rx=0 tx=0).stop 2024-09-12T01:33:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7fa7160ef640 1 -- 172.21.15.16:0/57381320 reap_dead start 2024-09-12T01:33:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f1b4b623640 1 -- 172.21.15.16:0/4294678582 shutdown_connections 2024-09-12T01:33:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7f1b4b623640 1 -- 172.21.15.16:0/4294678582 wait complete. 2024-09-12T01:33:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7f1b4b623640 1 Processor -- start 2024-09-12T01:33:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7f1b4b623640 1 -- start start 2024-09-12T01:33:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7fa717b52640 1 -- 172.21.15.16:0/57381320 shutdown_connections 2024-09-12T01:33:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7fa717b52640 1 -- 172.21.15.16:0/57381320 >> 172.21.15.16:0/57381320 conn(0x7fa70803a520 msgr2=0x7fa70803a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7f3a217fa640 1 -- 172.21.15.16:0/1483398464 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c014620 con 0x7f3a24163980 2024-09-12T01:33:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.806+0000 7f3a217fa640 1 -- 172.21.15.16:0/1483398464 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a1c019070 con 0x7f3a24163980 2024-09-12T01:33:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.806+0000 7f3a2af28640 1 -- 172.21.15.16:0/1483398464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a240c9970 con 0x7f3a24163980 2024-09-12T01:33:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.806+0000 7fa717b52640 1 -- 172.21.15.16:0/57381320 shutdown_connections 2024-09-12T01:33:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.806+0000 7f82d8cec640 1 -- 172.21.15.16:0/2654890647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc034020 con 0x7f82bc0052e0 2024-09-12T01:33:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.806+0000 7f3a2af28640 1 -- 172.21.15.16:0/1483398464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a240c9f30 con 0x7f3a24163980 2024-09-12T01:33:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.806+0000 7f3a217fa640 1 -- 172.21.15.16:0/1483398464 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c014a20 con 0x7f3a24163980 2024-09-12T01:33:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.806+0000 7fabdd7fa640 1 -- 172.21.15.16:0/3712574633 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4041020 con 0x7fabd8054140 2024-09-12T01:33:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.806+0000 7f84b6ffd640 1 -- 172.21.15.16:0/1637298507 <== osd.0 v2:172.21.15.16:6800/3659932612 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84bc043e50 con 0x7f84a80756a0 2024-09-12T01:33:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.806+0000 7fabdd7fa640 1 -- 172.21.15.16:0/3712574633 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabd40306e0 con 0x7fabd8054140 2024-09-12T01:33:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.807+0000 7f6cc359e640 1 Processor -- start 2024-09-12T01:33:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.807+0000 7f3a2af28640 1 -- 172.21.15.16:0/1483398464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a00001030 con 0x7f3a24163980 2024-09-12T01:33:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7feef8ff9640 1 -- 172.21.15.16:0/2588579860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef400fcf0 con 0x7feee4004800 2024-09-12T01:33:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7feef8ff9640 1 -- 172.21.15.16:0/2588579860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef401a070 con 0x7feee4004800 2024-09-12T01:33:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7feef8ff9640 1 -- 172.21.15.16:0/2588579860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef40156c0 con 0x7feee4004800 2024-09-12T01:33:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7fef02282640 1 -- 172.21.15.16:0/2588579860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc0c7ea0 con 0x7feee4004800 2024-09-12T01:33:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.803+0000 7fef02282640 1 -- 172.21.15.16:0/2588579860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc0c8460 con 0x7feee4004800 2024-09-12T01:33:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.805+0000 7feee27fc640 1 -- 172.21.15.16:0/2588579860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feefc0cb030 con 0x7feee4004800 2024-09-12T01:33:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.806+0000 7feef8ff9640 1 -- 172.21.15.16:0/2588579860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feef4015860 con 0x7feee4004800 2024-09-12T01:33:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.806+0000 7feef8ff9640 1 --2- 172.21.15.16:0/2588579860 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feee806bf00 0x7feee806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.807+0000 7feef8ff9640 1 -- 172.21.15.16:0/2588579860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7feef408bae0 con 0x7feee4004800 2024-09-12T01:33:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.807+0000 7feef8ff9640 1 --2- 172.21.15.16:0/2588579860 >> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] conn(0x7feee8075600 0x7feee8077a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.807+0000 7feef8ff9640 1 -- 172.21.15.16:0/2588579860 --> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feef4005ea0 con 0x7feee8075600 2024-09-12T01:33:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.806+0000 7fa717b52640 1 -- 172.21.15.16:0/57381320 wait complete. 2024-09-12T01:33:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.807+0000 7feefb7fe640 1 --2- 172.21.15.16:0/2588579860 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feee806bf00 0x7feee806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.807+0000 7fabdd7fa640 1 -- 172.21.15.16:0/3712574633 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4037040 con 0x7fabd8054140 2024-09-12T01:33:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.807+0000 7fabe59c5640 1 -- 172.21.15.16:0/3712574633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd80e8d00 con 0x7fabd8054140 2024-09-12T01:33:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.807+0000 7fabe59c5640 1 -- 172.21.15.16:0/3712574633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd80e8740 con 0x7fabd8054140 2024-09-12T01:33:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.807+0000 7f82dacf0640 1 -- 172.21.15.16:0/2654890647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc0052e0 msgr2=0x7f82bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.807+0000 7f82dacf0640 1 --2- 172.21.15.16:0/2654890647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc0052e0 0x7f82bc00b9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f82d415e9a0 tx=0x7f82bc005830 comp rx=0 tx=0).stop 2024-09-12T01:33:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f82dacf0640 1 -- 172.21.15.16:0/2654890647 shutdown_connections 2024-09-12T01:33:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f82dacf0640 1 --2- 172.21.15.16:0/2654890647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc0052e0 0x7f82bc00b9c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f82dacf0640 1 -- 172.21.15.16:0/2654890647 >> 172.21.15.16:0/2654890647 conn(0x7f82d40b1bd0 msgr2=0x7f82d40b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7fef0081f640 1 --2- 172.21.15.16:0/2588579860 >> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] conn(0x7feee8075600 0x7feee8077a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f1b4b623640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b30004d40 0x7f1b442092f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f1b4b623640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b441e4160 0x7f1b440b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f1b4b623640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b440b1d60 0x7f1b440b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7feefb7fe640 1 --2- 172.21.15.16:0/2588579860 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feee806bf00 0x7feee806e3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feef0039000 tx=0x7feef0010040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.807+0000 7f6cc359e640 1 -- start start 2024-09-12T01:33:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f1b493bf640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b30004d40 0x7f1b442092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f1b493bf640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b30004d40 0x7f1b442092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44410/0 (socket says 172.21.15.16:44410) 2024-09-12T01:33:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f1b4b623640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b440b4be0 con 0x7f1b441e4160 2024-09-12T01:33:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f1b4b623640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b440b4a60 con 0x7f1b440b1d60 2024-09-12T01:33:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f1b4b623640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f1b440d1860 con 0x7f1b30004d40 2024-09-12T01:33:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f1b493bf640 1 -- 172.21.15.16:0/3551502325 learned_addr learned my addr 172.21.15.16:0/3551502325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7fa717b52640 1 Processor -- start 2024-09-12T01:33:33.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7fa717b52640 1 -- start start 2024-09-12T01:33:33.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f82dacf0640 1 -- 172.21.15.16:0/2654890647 shutdown_connections 2024-09-12T01:33:33.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f1b48bbe640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b441e4160 0x7f1b440b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f3a217fa640 1 -- 172.21.15.16:0/1483398464 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a1c014bc0 con 0x7f3a24163980 2024-09-12T01:33:33.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f3a217fa640 1 --2- 172.21.15.16:0/1483398464 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f3a0c06bf00 0x7f3a0c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f3a217fa640 1 -- 172.21.15.16:0/1483398464 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3a1c08ba10 con 0x7f3a24163980 2024-09-12T01:33:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f3a217fa640 1 --2- 172.21.15.16:0/1483398464 >> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] conn(0x7f3a0c075600 0x7f3a0c077a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f3a217fa640 1 -- 172.21.15.16:0/1483398464 --> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a1c00ce50 con 0x7f3a0c075600 2024-09-12T01:33:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f3a217fa640 1 -- 172.21.15.16:0/1483398464 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3a1c092050 con 0x7f3a24163980 2024-09-12T01:33:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7fabe59c5640 1 -- 172.21.15.16:0/3712574633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fabd8053c00 con 0x7fabd8054140 2024-09-12T01:33:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f82dacf0640 1 -- 172.21.15.16:0/2654890647 wait complete. 2024-09-12T01:33:33.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7f1b49bc0640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b440b1d60 0x7f1b440b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.808+0000 7feef8ff9640 1 -- 172.21.15.16:0/2588579860 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feef4055cd0 con 0x7feee4004800 2024-09-12T01:33:33.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f1b48bbe640 1 -- 172.21.15.16:0/3551502325 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b30004d40 msgr2=0x7f1b442092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f1b48bbe640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b30004d40 0x7f1b442092f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f1b48bbe640 1 -- 172.21.15.16:0/3551502325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b440b1d60 msgr2=0x7f1b440b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f1b48bbe640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b440b1d60 0x7f1b440b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f1b48bbe640 1 -- 172.21.15.16:0/3551502325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4408bc80 con 0x7f1b441e4160 2024-09-12T01:33:33.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f1b493bf640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b30004d40 0x7f1b442092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f6cc359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac0037a0 0x7f6cc415a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f6cc359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc40cd4b0 0x7f6cc40cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f6cc359e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cc40b5b60 0x7f6cc40b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f6cc359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc40d2880 con 0x7f6cc40cd4b0 2024-09-12T01:33:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7fef0081f640 1 --2- 172.21.15.16:0/2588579860 >> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] conn(0x7feee8075600 0x7feee8077a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7fa717b52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70803e370 0x7fa708188520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7fa717b52640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708038600 0x7fa708188a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7fa717b52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa708188fa0 0x7fa708184f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7fa717b52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa708038070 con 0x7fa708188fa0 2024-09-12T01:33:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7fa717b52640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa708037ef0 con 0x7fa70803e370 2024-09-12T01:33:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7fa717b52640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa7080570d0 con 0x7fa708038600 2024-09-12T01:33:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7fa7150ed640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708038600 0x7fa708188a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7fa7150ed640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708038600 0x7fa708188a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44412/0 (socket says 172.21.15.16:44412) 2024-09-12T01:33:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7fa7150ed640 1 -- 172.21.15.16:0/446895211 learned_addr learned my addr 172.21.15.16:0/446895211 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f3a28cc4640 1 --2- 172.21.15.16:0/1483398464 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f3a0c06bf00 0x7f3a0c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.809+0000 7f6cc359e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc40d2700 con 0x7f6cac0037a0 2024-09-12T01:33:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f6cc359e640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc415e7d0 con 0x7f6cc40b5b60 2024-09-12T01:33:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f6cc259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac0037a0 0x7f6cc415a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f6cc259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac0037a0 0x7f6cc415a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45544/0 (socket says 172.21.15.16:45544) 2024-09-12T01:33:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f6cc259c640 1 -- 172.21.15.16:0/639989455 learned_addr learned my addr 172.21.15.16:0/639989455 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7fa7158ee640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70803e370 0x7fa708188520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f1b48bbe640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b441e4160 0x7f1b440b1820 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400e7f0 tx=0x7f1b3400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f3a23fff640 1 --2- 172.21.15.16:0/1483398464 >> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] conn(0x7f3a0c075600 0x7f3a0c077a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7fa7150ed640 1 -- 172.21.15.16:0/446895211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70803e370 msgr2=0x7fa708188520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7fa7150ed640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70803e370 0x7fa708188520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7fa7150ed640 1 -- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa708188fa0 msgr2=0x7fa708184f30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:33:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7fa7158ee640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa70803e370 0x7fa708188520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7fabdd7fa640 1 -- 172.21.15.16:0/3712574633 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fabd4044020 con 0x7fabd8054140 2024-09-12T01:33:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f1b39ffb640 1 -- 172.21.15.16:0/3551502325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34017070 con 0x7f1b441e4160 2024-09-12T01:33:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f1b39ffb640 1 -- 172.21.15.16:0/3551502325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3400c730 con 0x7f1b441e4160 2024-09-12T01:33:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f1b4b623640 1 -- 172.21.15.16:0/3551502325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b440b2680 con 0x7f1b441e4160 2024-09-12T01:33:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f6cc2d9d640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cc40b5b60 0x7f6cc40b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f6cc1d9b640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc40cd4b0 0x7f6cc40cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f1b39ffb640 1 -- 172.21.15.16:0/3551502325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3400cb20 con 0x7f1b441e4160 2024-09-12T01:33:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f3a28cc4640 1 --2- 172.21.15.16:0/1483398464 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f3a0c06bf00 0x7f3a0c06e3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009ac0 tx=0x7f3a14009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7fa7160ef640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa708188fa0 0x7fa708184f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.811+0000 7f3a23fff640 1 --2- 172.21.15.16:0/1483398464 >> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] conn(0x7f3a0c075600 0x7f3a0c077a40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f6cc2d9d640 1 -- 172.21.15.16:0/639989455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac0037a0 msgr2=0x7f6cc415a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f6cc2d9d640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac0037a0 0x7f6cc415a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f6cc2d9d640 1 -- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc40cd4b0 msgr2=0x7f6cc40cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7fa7150ed640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa708188fa0 0x7fa708184f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7fa7150ed640 1 -- 172.21.15.16:0/446895211 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70804d5a0 con 0x7fa708038600 2024-09-12T01:33:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.811+0000 7fa7160ef640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa708188fa0 0x7fa708184f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.811+0000 7fa7150ed640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708038600 0x7fa708188a60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa7100504d0 tx=0x7fa7100659b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.811+0000 7fabdd7fa640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fabc406bf30 0x7fabc406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.810+0000 7f1b4b623640 1 -- 172.21.15.16:0/3551502325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b441e3960 con 0x7f1b441e4160 2024-09-12T01:33:33.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.811+0000 7f82dacf0640 1 Processor -- start 2024-09-12T01:33:33.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.811+0000 7fabe49c3640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fabc406bf30 0x7fabc406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.811+0000 7f82dacf0640 1 -- start start 2024-09-12T01:33:33.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.811+0000 7fabdd7fa640 1 -- 172.21.15.16:0/3712574633 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fabd40ab820 con 0x7fabd8054140 2024-09-12T01:33:33.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7fabdd7fa640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] conn(0x7fabc40756b0 0x7fabc4077af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7fabdd7fa640 1 -- 172.21.15.16:0/3712574633 --> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fabd4032440 con 0x7fabc40756b0 2024-09-12T01:33:33.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7fabdd7fa640 1 -- 172.21.15.16:0/3712574633 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fabd40b02a0 con 0x7fabd8054140 2024-09-12T01:33:33.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7fabe49c3640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fabc406bf30 0x7fabc406e3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fabcc006a30 tx=0x7fabcc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.811+0000 7f6cc2d9d640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc40cd4b0 0x7f6cc40cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.811+0000 7f6cc2d9d640 1 -- 172.21.15.16:0/639989455 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc40ce5f0 con 0x7f6cc40b5b60 2024-09-12T01:33:33.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.811+0000 7f6cc1d9b640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc40cd4b0 0x7f6cc40cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:33.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.811+0000 7f6cc2d9d640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cc40b5b60 0x7f6cc40b5fe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40a2330 tx=0x7f6cbc00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7f6cb2ffd640 1 -- 172.21.15.16:0/639989455 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc00ed90 con 0x7f6cc40b5b60 2024-09-12T01:33:33.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7f6cb2ffd640 1 -- 172.21.15.16:0/639989455 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cbc019070 con 0x7f6cc40b5b60 2024-09-12T01:33:33.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7f6cb2ffd640 1 -- 172.21.15.16:0/639989455 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc014660 con 0x7f6cc40b5b60 2024-09-12T01:33:33.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7feef8ff9640 1 -- 172.21.15.16:0/2588579860 <== osd.4 v2:172.21.15.33:6819/1133469969 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feef4005ea0 con 0x7feee8075600 2024-09-12T01:33:33.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7f3a217fa640 1 -- 172.21.15.16:0/1483398464 <== osd.1 v2:172.21.15.16:6816/2624815404 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a1c00ce50 con 0x7f3a0c075600 2024-09-12T01:33:33.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7f6cc359e640 1 -- 172.21.15.16:0/639989455 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc40ccb30 con 0x7f6cc40b5b60 2024-09-12T01:33:33.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7f6cc359e640 1 -- 172.21.15.16:0/639989455 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc40cdd50 con 0x7f6cc40b5b60 2024-09-12T01:33:33.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7fa7067fc640 1 -- 172.21.15.16:0/446895211 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7100b1e20 con 0x7fa708038600 2024-09-12T01:33:33.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7fa7067fc640 1 -- 172.21.15.16:0/446895211 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa71006bca0 con 0x7fa708038600 2024-09-12T01:33:33.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7fabe51c4640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] conn(0x7fabc40756b0 0x7fabc4077af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.812+0000 7f1b4b623640 1 -- 172.21.15.16:0/3551502325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b440d1c30 con 0x7f1b441e4160 2024-09-12T01:33:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.813+0000 7fa7067fc640 1 -- 172.21.15.16:0/446895211 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71006be50 con 0x7fa708038600 2024-09-12T01:33:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.813+0000 7f1b39ffb640 1 -- 172.21.15.16:0/3551502325 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b3401a070 con 0x7f1b441e4160 2024-09-12T01:33:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.813+0000 7f6cc359e640 1 -- 172.21.15.16:0/639989455 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6cc41600d0 con 0x7f6cc40b5b60 2024-09-12T01:33:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.813+0000 7fa717b52640 1 -- 172.21.15.16:0/446895211 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7080eb050 con 0x7fa708038600 2024-09-12T01:33:33.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7fa717b52640 1 -- 172.21.15.16:0/446895211 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa708006d90 con 0x7fa708038600 2024-09-12T01:33:33.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7fabe51c4640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] conn(0x7fabc40756b0 0x7fabc4077af0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f1b39ffb640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1b2806be30 0x7f1b2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f82dacf0640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f82bc0052e0 0x7f82d41612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f1b39ffb640 1 -- 172.21.15.16:0/3551502325 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1b3408b690 con 0x7f1b441e4160 2024-09-12T01:33:33.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f82dacf0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d4160030 0x7f82d4156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f82dacf0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82d414a430 0x7f82d414a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f82dacf0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82d406e960 con 0x7f82d414a430 2024-09-12T01:33:33.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f82dacf0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82d406e7e0 con 0x7f82d4160030 2024-09-12T01:33:33.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f82dacf0640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f82d40aceb0 con 0x7f82bc0052e0 2024-09-12T01:33:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f1b39ffb640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] conn(0x7f1b28075530 0x7f1b28077970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f1b39ffb640 1 -- 172.21.15.16:0/3551502325 --> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b34013e50 con 0x7f1b28075530 2024-09-12T01:33:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f1b39ffb640 1 -- 172.21.15.16:0/3551502325 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b34092050 con 0x7f1b441e4160 2024-09-12T01:33:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f82d9cee640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f82bc0052e0 0x7f82d41612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f82d9cee640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f82bc0052e0 0x7f82d41612e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44416/0 (socket says 172.21.15.16:44416) 2024-09-12T01:33:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f82d9cee640 1 -- 172.21.15.16:0/2497441886 learned_addr learned my addr 172.21.15.16:0/2497441886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f1b493bf640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1b2806be30 0x7f1b2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.814+0000 7f82d94ed640 1 --2- 172.21.15.16:0/2497441886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d4160030 0x7f82d4156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7f1b49bc0640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] conn(0x7f1b28075530 0x7f1b28077970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7fa717b52640 1 -- 172.21.15.16:0/446895211 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa7080068c0 con 0x7fa708038600 2024-09-12T01:33:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7f82d9cee640 1 -- 172.21.15.16:0/2497441886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d4160030 msgr2=0x7f82d4156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7f82d9cee640 1 --2- 172.21.15.16:0/2497441886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d4160030 0x7f82d4156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7f82d9cee640 1 -- 172.21.15.16:0/2497441886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82d414a430 msgr2=0x7f82d414a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7f82d9cee640 1 --2- 172.21.15.16:0/2497441886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82d414a430 0x7f82d414a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7f82d9cee640 1 -- 172.21.15.16:0/2497441886 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d414adf0 con 0x7f82bc0052e0 2024-09-12T01:33:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7f82d9cee640 1 --2- 172.21.15.16:0/2497441886 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f82bc0052e0 0x7f82d41612e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f82c400f930 tx=0x7f82c400fdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7f1b493bf640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1b2806be30 0x7f1b2806e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009fd0 tx=0x7f1b40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7f82d27fc640 1 -- 172.21.15.16:0/2497441886 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c40052e0 con 0x7f82bc0052e0 2024-09-12T01:33:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7fabdd7fa640 1 -- 172.21.15.16:0/3712574633 <== osd.5 v2:172.21.15.33:6808/3986213904 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabd4032440 con 0x7fabc40756b0 2024-09-12T01:33:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7f82d27fc640 1 -- 172.21.15.16:0/2497441886 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c4018070 con 0x7f82bc0052e0 2024-09-12T01:33:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7f82dacf0640 1 -- 172.21.15.16:0/2497441886 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d41557b0 con 0x7f82bc0052e0 2024-09-12T01:33:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.815+0000 7f82d27fc640 1 -- 172.21.15.16:0/2497441886 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c400d9d0 con 0x7f82bc0052e0 2024-09-12T01:33:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.816+0000 7f82dacf0640 1 -- 172.21.15.16:0/2497441886 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d415f830 con 0x7f82bc0052e0 2024-09-12T01:33:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.816+0000 7f6cb2ffd640 1 -- 172.21.15.16:0/639989455 <== mon.2 v2:172.21.15.33:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6cbc005d10 con 0x7f6cc40b5b60 2024-09-12T01:33:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.816+0000 7fa7067fc640 1 -- 172.21.15.16:0/446895211 <== mon.2 v2:172.21.15.33:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa7100c3060 con 0x7fa708038600 2024-09-12T01:33:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.816+0000 7f1b49bc0640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] conn(0x7f1b28075530 0x7f1b28077970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.816+0000 7f6cb2ffd640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6ca406bf00 0x7f6ca406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.817+0000 7f6cc259c640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6ca406bf00 0x7f6ca406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.816+0000 7fa7067fc640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa6f406c050 0x7fa6f406e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.817+0000 7fa7067fc640 1 -- 172.21.15.16:0/446895211 <== mon.2 v2:172.21.15.33:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa710126850 con 0x7fa708038600 2024-09-12T01:33:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.817+0000 7fa7067fc640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] conn(0x7fa6f40757c0 0x7fa6f4077c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.817+0000 7fa7067fc640 1 -- 172.21.15.16:0/446895211 --> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa71006ee50 con 0x7fa6f40757c0 2024-09-12T01:33:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.817+0000 7fa7067fc640 1 -- 172.21.15.16:0/446895211 <== mon.2 v2:172.21.15.33:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa71012b2a0 con 0x7fa708038600 2024-09-12T01:33:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.817+0000 7f6cb2ffd640 1 -- 172.21.15.16:0/639989455 <== mon.2 v2:172.21.15.33:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6cbc08b190 con 0x7f6cc40b5b60 2024-09-12T01:33:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.817+0000 7f82dacf0640 1 -- 172.21.15.16:0/2497441886 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f82d40733b0 con 0x7f82bc0052e0 2024-09-12T01:33:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.817+0000 7f1b39ffb640 1 -- 172.21.15.16:0/3551502325 <== osd.7 v2:172.21.15.33:6800/2262537058 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b34013e50 con 0x7f1b28075530 2024-09-12T01:33:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.817+0000 7fa7158ee640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa6f406c050 0x7fa6f406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.818+0000 7fa7160ef640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] conn(0x7fa6f40757c0 0x7fa6f4077c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.818+0000 7f82d27fc640 1 -- 172.21.15.16:0/2497441886 <== mon.2 v2:172.21.15.33:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82c400db70 con 0x7f82bc0052e0 2024-09-12T01:33:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.819+0000 7fa7158ee640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa6f406c050 0x7fa6f406e510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa708038ad0 tx=0x7fa70c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.818+0000 7f82d27fc640 1 --2- 172.21.15.16:0/2497441886 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f82c006be80 0x7f82c006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.819+0000 7fa7160ef640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] conn(0x7fa6f40757c0 0x7fa6f4077c00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.817+0000 7f6cb2ffd640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] conn(0x7f6ca4075680 0x7f6ca4077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.817+0000 7f6cb2ffd640 1 -- 172.21.15.16:0/639989455 --> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6cbc00df30 con 0x7f6ca4075680 2024-09-12T01:33:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.817+0000 7f6cb2ffd640 1 -- 172.21.15.16:0/639989455 <== mon.2 v2:172.21.15.33:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6cbc092050 con 0x7f6cc40b5b60 2024-09-12T01:33:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.817+0000 7f6cc1d9b640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] conn(0x7f6ca4075680 0x7f6ca4077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.818+0000 7f6cc1d9b640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] conn(0x7f6ca4075680 0x7f6ca4077ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.819+0000 7f82d27fc640 1 -- 172.21.15.16:0/2497441886 <== mon.2 v2:172.21.15.33:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f82c408b170 con 0x7f82bc0052e0 2024-09-12T01:33:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.819+0000 7f82d27fc640 1 --2- 172.21.15.16:0/2497441886 >> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] conn(0x7f82c0075600 0x7f82c0077a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.819+0000 7f82d27fc640 1 -- 172.21.15.16:0/2497441886 --> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82c4009e50 con 0x7f82c0075600 2024-09-12T01:33:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.819+0000 7f82d27fc640 1 -- 172.21.15.16:0/2497441886 <== mon.2 v2:172.21.15.33:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f82c4092050 con 0x7f82bc0052e0 2024-09-12T01:33:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.820+0000 7f6cc259c640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6ca406bf00 0x7f6ca406e3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40012b0 tx=0x7f6cb4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.819+0000 7f82d94ed640 1 --2- 172.21.15.16:0/2497441886 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f82c006be80 0x7f82c006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.819+0000 7f82da4ef640 1 --2- 172.21.15.16:0/2497441886 >> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] conn(0x7f82c0075600 0x7f82c0077a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.820+0000 7f6cb2ffd640 1 -- 172.21.15.16:0/639989455 <== osd.2 v2:172.21.15.16:6824/3738115042 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6cbc00df30 con 0x7f6ca4075680 2024-09-12T01:33:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.820+0000 7f82d94ed640 1 --2- 172.21.15.16:0/2497441886 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f82c006be80 0x7f82c006e340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f82d41f7500 tx=0x7f82c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.821+0000 7f82da4ef640 1 --2- 172.21.15.16:0/2497441886 >> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] conn(0x7f82c0075600 0x7f82c0077a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.821+0000 7fa7067fc640 1 -- 172.21.15.16:0/446895211 <== osd.3 v2:172.21.15.16:6808/1588628599 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa71006ee50 con 0x7fa6f40757c0 2024-09-12T01:33:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.821+0000 7f82d27fc640 1 -- 172.21.15.16:0/2497441886 <== osd.6 v2:172.21.15.33:6816/1471795647 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f82c4009e50 con 0x7f82c0075600 2024-09-12T01:33:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.832+0000 7feee27fc640 1 -- 172.21.15.16:0/2588579860 --> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feefc06d740 con 0x7feee8075600 2024-09-12T01:33:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.832+0000 7feef8ff9640 1 -- 172.21.15.16:0/2588579860 <== osd.4 v2:172.21.15.33:6819/1133469969 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feefc06d740 con 0x7feee8075600 2024-09-12T01:33:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.833+0000 7fef02282640 1 -- 172.21.15.16:0/2588579860 >> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] conn(0x7feee8075600 msgr2=0x7feee8077a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.833+0000 7fef02282640 1 --2- 172.21.15.16:0/2588579860 >> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] conn(0x7feee8075600 0x7feee8077a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.833+0000 7fef02282640 1 -- 172.21.15.16:0/2588579860 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feee806bf00 msgr2=0x7feee806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.833+0000 7fef02282640 1 --2- 172.21.15.16:0/2588579860 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feee806bf00 0x7feee806e3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feef0039000 tx=0x7feef0010040 comp rx=0 tx=0).stop 2024-09-12T01:33:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.834+0000 7fef02282640 1 -- 172.21.15.16:0/2588579860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee4004800 msgr2=0x7feefc1597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.834+0000 7fef02282640 1 --2- 172.21.15.16:0/2588579860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee4004800 0x7feefc1597f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feef4006730 tx=0x7feef400f530 comp rx=0 tx=0).stop 2024-09-12T01:33:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.834+0000 7fef0081f640 1 -- 172.21.15.16:0/2588579860 reap_dead start 2024-09-12T01:33:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.834+0000 7fef02282640 1 -- 172.21.15.16:0/2588579860 shutdown_connections 2024-09-12T01:33:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.834+0000 7fef02282640 1 -- 172.21.15.16:0/2588579860 >> 172.21.15.16:0/2588579860 conn(0x7feefc0b1af0 msgr2=0x7feefc073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.834+0000 7fef02282640 1 -- 172.21.15.16:0/2588579860 shutdown_connections 2024-09-12T01:33:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.835+0000 7fef02282640 1 -- 172.21.15.16:0/2588579860 wait complete. 2024-09-12T01:33:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.837+0000 7f84cca30640 1 -- 172.21.15.16:0/1637298507 --> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f84c80b5f20 con 0x7f84a80756a0 2024-09-12T01:33:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.838+0000 7f84b6ffd640 1 -- 172.21.15.16:0/1637298507 <== osd.0 v2:172.21.15.16:6800/3659932612 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f84c80b5f20 con 0x7f84a80756a0 2024-09-12T01:33:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.839+0000 7f84b4ff9640 1 -- 172.21.15.16:0/1637298507 >> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] conn(0x7f84a80756a0 msgr2=0x7f84a8077ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.839+0000 7f84b4ff9640 1 --2- 172.21.15.16:0/1637298507 >> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] conn(0x7f84a80756a0 0x7f84a8077ae0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.839+0000 7f84b4ff9640 1 -- 172.21.15.16:0/1637298507 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f84a806bf30 msgr2=0x7f84a806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.839+0000 7f84b4ff9640 1 --2- 172.21.15.16:0/1637298507 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f84a806bf30 0x7f84a806e3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f84c00012e0 tx=0x7f84c0006290 comp rx=0 tx=0).stop 2024-09-12T01:33:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.839+0000 7f84b4ff9640 1 -- 172.21.15.16:0/1637298507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c80cd360 msgr2=0x7f84c80ae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.839+0000 7f84b4ff9640 1 --2- 172.21.15.16:0/1637298507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c80cd360 0x7f84c80ae4a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84bc02ed30 tx=0x7f84bc0046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.839+0000 7f84c6d9d640 1 -- 172.21.15.16:0/1637298507 reap_dead start 2024-09-12T01:33:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.840+0000 7f84b4ff9640 1 -- 172.21.15.16:0/1637298507 shutdown_connections 2024-09-12T01:33:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.840+0000 7f84b4ff9640 1 -- 172.21.15.16:0/1637298507 >> 172.21.15.16:0/1637298507 conn(0x7f84c80b1af0 msgr2=0x7f84c80b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.840+0000 7f84b4ff9640 1 -- 172.21.15.16:0/1637298507 shutdown_connections 2024-09-12T01:33:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.840+0000 7f84b4ff9640 1 -- 172.21.15.16:0/1637298507 wait complete. 2024-09-12T01:33:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.845+0000 7f3a2af28640 1 -- 172.21.15.16:0/1483398464 --> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3a00002dd0 con 0x7f3a0c075600 2024-09-12T01:33:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.845+0000 7f3a217fa640 1 -- 172.21.15.16:0/1483398464 <== osd.1 v2:172.21.15.16:6816/2624815404 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3a00002dd0 con 0x7f3a0c075600 2024-09-12T01:33:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.846+0000 7f3a1affd640 1 -- 172.21.15.16:0/1483398464 >> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] conn(0x7f3a0c075600 msgr2=0x7f3a0c077a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.846+0000 7f3a1affd640 1 --2- 172.21.15.16:0/1483398464 >> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] conn(0x7f3a0c075600 0x7f3a0c077a40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.846+0000 7f3a1affd640 1 -- 172.21.15.16:0/1483398464 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f3a0c06bf00 msgr2=0x7f3a0c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.846+0000 7f3a1affd640 1 --2- 172.21.15.16:0/1483398464 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f3a0c06bf00 0x7f3a0c06e3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009ac0 tx=0x7f3a14009190 comp rx=0 tx=0).stop 2024-09-12T01:33:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.847+0000 7f3a1affd640 1 -- 172.21.15.16:0/1483398464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a24163980 msgr2=0x7f3a24151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.847+0000 7f3a1affd640 1 --2- 172.21.15.16:0/1483398464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a24163980 0x7f3a24151350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c007e40 tx=0x7f3a1c00e4b0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.847+0000 7f3a294c5640 1 -- 172.21.15.16:0/1483398464 reap_dead start 2024-09-12T01:33:33.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.847+0000 7f3a1affd640 1 -- 172.21.15.16:0/1483398464 shutdown_connections 2024-09-12T01:33:33.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.847+0000 7f3a1affd640 1 -- 172.21.15.16:0/1483398464 >> 172.21.15.16:0/1483398464 conn(0x7f3a240cffa0 msgr2=0x7f3a24163e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.846+0000 7fabe59c5640 1 -- 172.21.15.16:0/3712574633 --> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fabd8037f80 con 0x7fabc40756b0 2024-09-12T01:33:33.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.847+0000 7fabdd7fa640 1 -- 172.21.15.16:0/3712574633 <== osd.5 v2:172.21.15.33:6808/3986213904 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fabd8037f80 con 0x7fabc40756b0 2024-09-12T01:33:33.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.847+0000 7f3a1affd640 1 -- 172.21.15.16:0/1483398464 shutdown_connections 2024-09-12T01:33:33.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.847+0000 7f3a1affd640 1 -- 172.21.15.16:0/1483398464 wait complete. 2024-09-12T01:33:33.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.848+0000 7fabe59c5640 1 -- 172.21.15.16:0/3712574633 >> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] conn(0x7fabc40756b0 msgr2=0x7fabc4077af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.848+0000 7fabe59c5640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] conn(0x7fabc40756b0 0x7fabc4077af0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.848+0000 7fabe59c5640 1 -- 172.21.15.16:0/3712574633 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fabc406bf30 msgr2=0x7fabc406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.848+0000 7fabe59c5640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fabc406bf30 0x7fabc406e3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fabcc006a30 tx=0x7fabcc008040 comp rx=0 tx=0).stop 2024-09-12T01:33:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.848+0000 7fabe59c5640 1 -- 172.21.15.16:0/3712574633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd8054140 msgr2=0x7fabd8007620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.848+0000 7fabe59c5640 1 --2- 172.21.15.16:0/3712574633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd8054140 0x7fabd8007620 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fabd40098a0 tx=0x7fabd40046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.848+0000 7fabe51c4640 1 -- 172.21.15.16:0/3712574633 reap_dead start 2024-09-12T01:33:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.848+0000 7fabe59c5640 1 -- 172.21.15.16:0/3712574633 shutdown_connections 2024-09-12T01:33:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.848+0000 7fabe59c5640 1 -- 172.21.15.16:0/3712574633 >> 172.21.15.16:0/3712574633 conn(0x7fabd801ea40 msgr2=0x7fabd8033c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.849+0000 7fabe59c5640 1 -- 172.21.15.16:0/3712574633 shutdown_connections 2024-09-12T01:33:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.849+0000 7fabe59c5640 1 -- 172.21.15.16:0/3712574633 wait complete. 2024-09-12T01:33:33.905 INFO:teuthology.orchestra.run.smithi016.stdout:64424509451 2024-09-12T01:33:33.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T01:33:33.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.852+0000 7fa717b52640 1 -- 172.21.15.16:0/446895211 --> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa7080567a0 con 0x7fa6f40757c0 2024-09-12T01:33:33.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.852+0000 7f6cc359e640 1 -- 172.21.15.16:0/639989455 --> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6cc40ce1e0 con 0x7f6ca4075680 2024-09-12T01:33:33.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.853+0000 7fa7067fc640 1 -- 172.21.15.16:0/446895211 <== osd.3 v2:172.21.15.16:6808/1588628599 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa7080567a0 con 0x7fa6f40757c0 2024-09-12T01:33:33.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.853+0000 7f6cb2ffd640 1 -- 172.21.15.16:0/639989455 <== osd.2 v2:172.21.15.16:6824/3738115042 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6cc40ce1e0 con 0x7f6ca4075680 2024-09-12T01:33:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.853+0000 7fa717b52640 1 -- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] conn(0x7fa6f40757c0 msgr2=0x7fa6f4077c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7fa717b52640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] conn(0x7fa6f40757c0 0x7fa6f4077c00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7f6cc359e640 1 -- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] conn(0x7f6ca4075680 msgr2=0x7f6ca4077ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7f6cc359e640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] conn(0x7f6ca4075680 0x7f6ca4077ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7f6cc359e640 1 -- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6ca406bf00 msgr2=0x7f6ca406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7f6cc359e640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6ca406bf00 0x7f6ca406e3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40012b0 tx=0x7f6cb4009040 comp rx=0 tx=0).stop 2024-09-12T01:33:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7fa717b52640 1 -- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa6f406c050 msgr2=0x7fa6f406e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7fa717b52640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa6f406c050 0x7fa6f406e510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa708038ad0 tx=0x7fa70c009040 comp rx=0 tx=0).stop 2024-09-12T01:33:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7fa717b52640 1 -- 172.21.15.16:0/446895211 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708038600 msgr2=0x7fa708188a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7fa717b52640 1 --2- 172.21.15.16:0/446895211 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa708038600 0x7fa708188a60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa7100504d0 tx=0x7fa7100659b0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7f6cc359e640 1 -- 172.21.15.16:0/639989455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cc40b5b60 msgr2=0x7f6cc40b5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7f6cc359e640 1 --2- 172.21.15.16:0/639989455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6cc40b5b60 0x7f6cc40b5fe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40a2330 tx=0x7f6cbc00e4b0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7fa7160ef640 1 -- 172.21.15.16:0/446895211 reap_dead start 2024-09-12T01:33:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7f6cc2d9d640 1 -- 172.21.15.16:0/639989455 reap_dead start 2024-09-12T01:33:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7fa717b52640 1 -- 172.21.15.16:0/446895211 shutdown_connections 2024-09-12T01:33:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7f6cc359e640 1 -- 172.21.15.16:0/639989455 shutdown_connections 2024-09-12T01:33:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7f6cc359e640 1 -- 172.21.15.16:0/639989455 >> 172.21.15.16:0/639989455 conn(0x7f6cc408ddd0 msgr2=0x7f6cc40b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7fa717b52640 1 -- 172.21.15.16:0/446895211 >> 172.21.15.16:0/446895211 conn(0x7fa70803a520 msgr2=0x7fa7080e4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7fa717b52640 1 -- 172.21.15.16:0/446895211 shutdown_connections 2024-09-12T01:33:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.854+0000 7fa717b52640 1 -- 172.21.15.16:0/446895211 wait complete. 2024-09-12T01:33:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.855+0000 7f6cc359e640 1 -- 172.21.15.16:0/639989455 shutdown_connections 2024-09-12T01:33:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.855+0000 7f82dacf0640 1 -- 172.21.15.16:0/2497441886 --> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f82d40b5140 con 0x7f82c0075600 2024-09-12T01:33:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.855+0000 7f6cc359e640 1 -- 172.21.15.16:0/639989455 wait complete. 2024-09-12T01:33:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.855+0000 7f82d27fc640 1 -- 172.21.15.16:0/2497441886 <== osd.6 v2:172.21.15.33:6816/1471795647 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f82d40b5140 con 0x7f82c0075600 2024-09-12T01:33:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.856+0000 7f82dacf0640 1 -- 172.21.15.16:0/2497441886 >> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] conn(0x7f82c0075600 msgr2=0x7f82c0077a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.856+0000 7f82dacf0640 1 --2- 172.21.15.16:0/2497441886 >> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] conn(0x7f82c0075600 0x7f82c0077a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.856+0000 7f82dacf0640 1 -- 172.21.15.16:0/2497441886 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f82c006be80 msgr2=0x7f82c006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.856+0000 7f82dacf0640 1 --2- 172.21.15.16:0/2497441886 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f82c006be80 0x7f82c006e340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f82d41f7500 tx=0x7f82c8008040 comp rx=0 tx=0).stop 2024-09-12T01:33:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.856+0000 7f82dacf0640 1 -- 172.21.15.16:0/2497441886 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f82bc0052e0 msgr2=0x7f82d41612e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.856+0000 7f82dacf0640 1 --2- 172.21.15.16:0/2497441886 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f82bc0052e0 0x7f82d41612e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f82c400f930 tx=0x7f82c400fdf0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.856+0000 7f82da4ef640 1 -- 172.21.15.16:0/2497441886 reap_dead start 2024-09-12T01:33:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.856+0000 7f82dacf0640 1 -- 172.21.15.16:0/2497441886 shutdown_connections 2024-09-12T01:33:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.856+0000 7f82dacf0640 1 -- 172.21.15.16:0/2497441886 >> 172.21.15.16:0/2497441886 conn(0x7f82d40b1bd0 msgr2=0x7f82d40b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.857+0000 7f82dacf0640 1 -- 172.21.15.16:0/2497441886 shutdown_connections 2024-09-12T01:33:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.857+0000 7f82dacf0640 1 -- 172.21.15.16:0/2497441886 wait complete. 2024-09-12T01:33:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.857+0000 7f1b4b623640 1 -- 172.21.15.16:0/3551502325 --> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b440d2970 con 0x7f1b28075530 2024-09-12T01:33:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.857+0000 7f1b39ffb640 1 -- 172.21.15.16:0/3551502325 <== osd.7 v2:172.21.15.33:6800/2262537058 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1b440d2970 con 0x7f1b28075530 2024-09-12T01:33:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.858+0000 7f1b4b623640 1 -- 172.21.15.16:0/3551502325 >> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] conn(0x7f1b28075530 msgr2=0x7f1b28077970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.858+0000 7f1b4b623640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] conn(0x7f1b28075530 0x7f1b28077970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.858+0000 7f1b4b623640 1 -- 172.21.15.16:0/3551502325 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1b2806be30 msgr2=0x7f1b2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.858+0000 7f1b4b623640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1b2806be30 0x7f1b2806e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009fd0 tx=0x7f1b40008040 comp rx=0 tx=0).stop 2024-09-12T01:33:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.858+0000 7f1b4b623640 1 -- 172.21.15.16:0/3551502325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b441e4160 msgr2=0x7f1b440b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.858+0000 7f1b4b623640 1 --2- 172.21.15.16:0/3551502325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b441e4160 0x7f1b440b1820 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400e7f0 tx=0x7f1b3400ecb0 comp rx=0 tx=0).stop 2024-09-12T01:33:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.858+0000 7f1b49bc0640 1 -- 172.21.15.16:0/3551502325 reap_dead start 2024-09-12T01:33:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.859+0000 7f1b4b623640 1 -- 172.21.15.16:0/3551502325 shutdown_connections 2024-09-12T01:33:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.859+0000 7f1b4b623640 1 -- 172.21.15.16:0/3551502325 >> 172.21.15.16:0/3551502325 conn(0x7f1b4409d7c0 msgr2=0x7f1b440cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.859+0000 7f1b4b623640 1 -- 172.21.15.16:0/3551502325 shutdown_connections 2024-09-12T01:33:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:33.859+0000 7f1b4b623640 1 -- 172.21.15.16:0/3551502325 wait complete. 2024-09-12T01:33:33.916 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2024-09-12T01:33:33.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T01:33:33.917 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2024-09-12T01:33:33.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T01:33:33.918 INFO:teuthology.orchestra.run.smithi016.stdout:60129542153 2024-09-12T01:33:33.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T01:33:33.919 INFO:teuthology.orchestra.run.smithi016.stdout:64424509451 2024-09-12T01:33:33.920 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T01:33:33.920 INFO:teuthology.orchestra.run.smithi016.stdout:60129542154 2024-09-12T01:33:33.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T01:33:33.921 INFO:teuthology.orchestra.run.smithi016.stdout:55834574858 2024-09-12T01:33:33.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T01:33:33.922 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2024-09-12T01:33:33.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T01:33:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.131+0000 7f0e3e381640 1 Processor -- start 2024-09-12T01:33:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.131+0000 7f0e3e381640 1 -- start start 2024-09-12T01:33:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.131+0000 7f849aa45640 1 Processor -- start 2024-09-12T01:33:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.132+0000 7f68f550a640 1 Processor -- start 2024-09-12T01:33:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f0e3e381640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e380ccf80 0x7f0e380af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f0e3e381640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e380af5d0 0x7f0e38159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.131+0000 7f849aa45640 1 -- start start 2024-09-12T01:33:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f849aa45640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8494162740 0x7f84940b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f849aa45640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f84940c5670 0x7f84940d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f0e3c91e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0e380cd360 0x7f0e3806da70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53644/0 (socket says 172.21.15.16:53644) 2024-09-12T01:33:34.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f849aa45640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84940cfc00 con 0x7f8494166cb0 2024-09-12T01:33:34.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f849aa45640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f849406cd80 con 0x7f84940c5670 2024-09-12T01:33:34.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f849aa45640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f849406d060 con 0x7f8494162740 2024-09-12T01:33:34.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f84937fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8494166cb0 0x7f84940c4f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53632/0 (socket says 172.21.15.16:53632) 2024-09-12T01:33:34.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f0e377fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e380af5d0 0x7f0e38159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f84937fe640 1 -- 172.21.15.16:0/1000656319 learned_addr learned my addr 172.21.15.16:0/1000656319 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:34.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f0e377fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e380af5d0 0x7f0e38159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45562/0 (socket says 172.21.15.16:45562) 2024-09-12T01:33:34.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f0e37fff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e380ccf80 0x7f0e380af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f0e37fff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e380ccf80 0x7f0e380af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44420/0 (socket says 172.21.15.16:44420) 2024-09-12T01:33:34.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f0e3c91e640 1 -- 172.21.15.16:0/4159325052 learned_addr learned my addr 172.21.15.16:0/4159325052 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:34.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f8493fff640 1 --2- 172.21.15.16:0/1000656319 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8494162740 0x7f84940b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f8498fe2640 1 --2- 172.21.15.16:0/1000656319 >> v2:172.21.15.33:3300/0 conn(0x7f84940c5670 0x7f84940d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.134+0000 7f0e3e381640 1 -- 172.21.15.16:0/4159325052 --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e3815e260 con 0x7f0e380cd360 2024-09-12T01:33:34.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.132+0000 7f68f550a640 1 -- start start 2024-09-12T01:33:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f68f550a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68e800ba50 0x7f68e8008b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f68f550a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f68e8037890 0x7f68e8054130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f68f550a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68e80322b0 con 0x7f68e800be30 2024-09-12T01:33:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f68f550a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f68e8034e80 con 0x7f68e8037890 2024-09-12T01:33:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f68f550a640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f68e800acb0 con 0x7f68e800ba50 2024-09-12T01:33:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f68ee7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f68e800be30 0x7f68e8037180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53652/0 (socket says 172.21.15.16:53652) 2024-09-12T01:33:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.133+0000 7f68ee7fc640 1 -- 172.21.15.16:0/2323872039 learned_addr learned my addr 172.21.15.16:0/2323872039 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.134+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 594302 0 0) 0x7f68e80322b0 con 0x7f68e800be30 2024-09-12T01:33:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.134+0000 7f0e3e381640 1 -- 172.21.15.16:0/4159325052 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f0e3815ede0 con 0x7f0e380af5d0 2024-09-12T01:33:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.134+0000 7f0e3e381640 1 -- 172.21.15.16:0/4159325052 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3815e540 con 0x7f0e380ccf80 2024-09-12T01:33:34.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.134+0000 7f68eeffd640 1 --2- 172.21.15.16:0/2323872039 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68e800ba50 0x7f68e8008b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.134+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68d8003520 con 0x7f68e800be30 2024-09-12T01:33:34.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1328672705 0 0) 0x7f68d8003520 con 0x7f68e800be30 2024-09-12T01:33:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f0e377fe640 1 -- 172.21.15.16:0/4159325052 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e380ccf80 msgr2=0x7f0e380af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f0e36ffd640 1 -- 172.21.15.16:0/4159325052 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3250345038 0 0) 0x7f0e3815e260 con 0x7f0e380cd360 2024-09-12T01:33:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f0e377fe640 1 --2- 172.21.15.16:0/4159325052 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e380ccf80 0x7f0e380af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f0e377fe640 1 -- 172.21.15.16:0/4159325052 >> v1:172.21.15.16:6789/0 conn(0x7f0e380cd360 legacy=0x7f0e3806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68ef7fe640 1 --2- 172.21.15.16:0/2323872039 >> v2:172.21.15.33:3300/0 conn(0x7f68e8037890 0x7f68e8054130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f8f7d01e640 1 Processor -- start 2024-09-12T01:33:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7fd741fbd640 1 Processor -- start 2024-09-12T01:33:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f0e377fe640 1 -- 172.21.15.16:0/4159325052 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3815e820 con 0x7f0e380af5d0 2024-09-12T01:33:34.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68e800ba50 msgr2=0x7f68e8008b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:34.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 --2- 172.21.15.16:0/2323872039 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68e800ba50 0x7f68e8008b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 >> v2:172.21.15.33:3300/0 conn(0x7f68e8037890 msgr2=0x7f68e8054130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:34.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 --2- 172.21.15.16:0/2323872039 >> v2:172.21.15.33:3300/0 conn(0x7f68e8037890 0x7f68e8054130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e800af90 con 0x7f68e800be30 2024-09-12T01:33:34.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f68f0050800 con 0x7f68e800be30 2024-09-12T01:33:34.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 >> v1:172.21.15.16:6789/0 conn(0x7f68e800be30 legacy=0x7f68e8037180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 --2- 172.21.15.16:0/2323872039 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68d80039b0 0x7f68d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f8f7d01e640 1 -- start start 2024-09-12T01:33:34.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 --2- 172.21.15.16:0/2323872039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68d8004350 0x7f68d8004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 --2- 172.21.15.16:0/2323872039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d8005020 0x7f68d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68e8034e80 con 0x7f68d8005020 2024-09-12T01:33:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68e800acb0 con 0x7f68d8004350 2024-09-12T01:33:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f68d800bfb0 con 0x7f68d80039b0 2024-09-12T01:33:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f68ef7fe640 1 --2- 172.21.15.16:0/2323872039 >> v2:172.21.15.33:3300/0 conn(0x7f68e8037890 0x7f68e8054130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7f0e37fff640 1 --2- 172.21.15.16:0/4159325052 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e380ccf80 0x7f0e380af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7f68ef7fe640 1 --2- 172.21.15.16:0/2323872039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d8005020 0x7f68d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7f68ef7fe640 1 -- 172.21.15.16:0/2323872039 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68d80039b0 msgr2=0x7f68d8003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:33:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7f68ef7fe640 1 --2- 172.21.15.16:0/2323872039 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68d80039b0 0x7f68d8003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7f68ef7fe640 1 -- 172.21.15.16:0/2323872039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68d8004350 msgr2=0x7f68d8004820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:33:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7f68ef7fe640 1 --2- 172.21.15.16:0/2323872039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68d8004350 0x7f68d8004820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7f68ef7fe640 1 -- 172.21.15.16:0/2323872039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e800af90 con 0x7f68d8005020 2024-09-12T01:33:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7f0e377fe640 1 --2- 172.21.15.16:0/4159325052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e380af5d0 0x7f0e38159e20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e280097a0 tx=0x7f0e2802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbc1805b4aa1c153 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f8493fff640 1 -- 172.21.15.16:0/1000656319 >> v2:172.21.15.33:3300/0 conn(0x7f84940c5670 msgr2=0x7f84940d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f8493fff640 1 --2- 172.21.15.16:0/1000656319 >> v2:172.21.15.33:3300/0 conn(0x7f84940c5670 0x7f84940d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7f8493fff640 1 -- 172.21.15.16:0/1000656319 >> v1:172.21.15.16:6789/0 conn(0x7f8494166cb0 legacy=0x7f84940c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7f8493fff640 1 -- 172.21.15.16:0/1000656319 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8494162b20 con 0x7f8494162740 2024-09-12T01:33:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7f8493fff640 1 --2- 172.21.15.16:0/1000656319 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8494162740 0x7f84940b2da0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f84880097a0 tx=0x7f848802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cbc46d6a6acde47 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7f8f7d01e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8f780ccf80 0x7f8f780af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7f8f7d01e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f780af5d0 0x7f8f780ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7f8f7d01e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f7815e260 con 0x7f8f780b25e0 2024-09-12T01:33:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7f8f7d01e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8f7815ede0 con 0x7f8f780af5d0 2024-09-12T01:33:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7f8f7d01e640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7815e540 con 0x7f8f780ccf80 2024-09-12T01:33:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7f8f7659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8f780b25e0 0x7f8f78159e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53672/0 (socket says 172.21.15.16:53672) 2024-09-12T01:33:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7f8f7659c640 1 -- 172.21.15.16:0/2216364084 learned_addr learned my addr 172.21.15.16:0/2216364084 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.135+0000 7fd741fbd640 1 -- start start 2024-09-12T01:33:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7fd741fbd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd73c15a4e0 0x7fd73c06da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7fd741fbd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73c15a8c0 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7fd741fbd640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73c06ea10 con 0x7fd73c0ce580 2024-09-12T01:33:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.136+0000 7fd741fbd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd73c0c5440 con 0x7fd73c15a4e0 2024-09-12T01:33:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7fd73bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd73c0cd4a0 0x7fd73c0c58d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53666/0 (socket says 172.21.15.16:53666) 2024-09-12T01:33:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7fd73bfff640 1 -- 172.21.15.16:0/2686339517 learned_addr learned my addr 172.21.15.16:0/2686339517 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7fd73b7fe640 1 --2- 172.21.15.16:0/2686339517 >> v2:172.21.15.16:3300/0 conn(0x7fd73c15a4e0 0x7fd73c06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7f8492ffd640 1 -- 172.21.15.16:0/1000656319 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488037070 con 0x7f8494162740 2024-09-12T01:33:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7f8492ffd640 1 -- 172.21.15.16:0/1000656319 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848802adc0 con 0x7f8494162740 2024-09-12T01:33:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7f8f7759e640 1 --2- 172.21.15.16:0/2216364084 >> v2:172.21.15.16:3300/0 conn(0x7f8f780af5d0 0x7f8f780ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7f8f76d9d640 1 --2- 172.21.15.16:0/2216364084 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8f780ccf80 0x7f8f780af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7f8492ffd640 1 -- 172.21.15.16:0/1000656319 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488032640 con 0x7f8494162740 2024-09-12T01:33:34.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7f68ef7fe640 1 --2- 172.21.15.16:0/2323872039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d8005020 0x7f68d800b9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f68e400aa70 tx=0x7f68d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1fa1c264edb776 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7fd73a7fc640 1 -- 172.21.15.16:0/2686339517 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2585471447 0 0) 0x7fd73c15a8c0 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7fd73a7fc640 1 -- 172.21.15.16:0/2686339517 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd724003520 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7fd73a7fc640 1 -- 172.21.15.16:0/2686339517 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1463577375 0 0) 0x7fd73c06ea10 con 0x7fd73c0ce580 2024-09-12T01:33:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.137+0000 7fd73a7fc640 1 -- 172.21.15.16:0/2686339517 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd73c15a8c0 con 0x7fd73c0ce580 2024-09-12T01:33:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.138+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e400a640 con 0x7f68d8005020 2024-09-12T01:33:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.138+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68e402acb0 con 0x7f68d8005020 2024-09-12T01:33:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.138+0000 7f68edffb640 1 -- 172.21.15.16:0/2323872039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4004240 con 0x7f68d8005020 2024-09-12T01:33:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.138+0000 7f0e36ffd640 1 -- 172.21.15.16:0/4159325052 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28037070 con 0x7f0e380af5d0 2024-09-12T01:33:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.138+0000 7f0e36ffd640 1 -- 172.21.15.16:0/4159325052 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e28032440 con 0x7f0e380af5d0 2024-09-12T01:33:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.138+0000 7f0e36ffd640 1 -- 172.21.15.16:0/4159325052 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28032850 con 0x7f0e380af5d0 2024-09-12T01:33:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f0e3e381640 1 -- 172.21.15.16:0/4159325052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e380af5d0 msgr2=0x7f0e38159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f0e3e381640 1 --2- 172.21.15.16:0/4159325052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e380af5d0 0x7f0e38159e20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e280097a0 tx=0x7f0e2802a1a0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f849aa45640 1 -- 172.21.15.16:0/1000656319 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8494162740 msgr2=0x7f84940b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f849aa45640 1 --2- 172.21.15.16:0/1000656319 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8494162740 0x7f84940b2da0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f84880097a0 tx=0x7f848802a130 comp rx=0 tx=0).stop 2024-09-12T01:33:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f849aa45640 1 -- 172.21.15.16:0/1000656319 shutdown_connections 2024-09-12T01:33:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f849aa45640 1 --2- 172.21.15.16:0/1000656319 >> v2:172.21.15.33:3300/0 conn(0x7f84940c5670 0x7f84940d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f849aa45640 1 --2- 172.21.15.16:0/1000656319 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8494162740 0x7f84940b2da0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f84880097a0 tx=0x7f848802a130 comp rx=0 tx=0).stop 2024-09-12T01:33:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f849aa45640 1 -- 172.21.15.16:0/1000656319 >> 172.21.15.16:0/1000656319 conn(0x7f849409b2e0 msgr2=0x7f849408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f8f7759e640 1 -- 172.21.15.16:0/2216364084 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8f780ccf80 msgr2=0x7f8f780af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f8f7759e640 1 --2- 172.21.15.16:0/2216364084 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8f780ccf80 0x7f8f780af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f8f7759e640 1 -- 172.21.15.16:0/2216364084 >> v1:172.21.15.16:6789/0 conn(0x7f8f780b25e0 legacy=0x7f8f78159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f8f7759e640 1 -- 172.21.15.16:0/2216364084 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7815e820 con 0x7f8f780af5d0 2024-09-12T01:33:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.138+0000 7fd73b7fe640 1 -- 172.21.15.16:0/2686339517 >> v1:172.21.15.33:6789/0 conn(0x7fd73c0ce580 legacy=0x7fd73c0ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.138+0000 7fd73b7fe640 1 -- 172.21.15.16:0/2686339517 >> v1:172.21.15.16:6789/0 conn(0x7fd73c0cd4a0 legacy=0x7fd73c0c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.138+0000 7fd73b7fe640 1 -- 172.21.15.16:0/2686339517 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c06e170 con 0x7fd73c15a4e0 2024-09-12T01:33:34.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.138+0000 7fcb3988b640 1 Processor -- start 2024-09-12T01:33:34.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.138+0000 7f5c4b427640 1 Processor -- start 2024-09-12T01:33:34.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.138+0000 7f5c4b427640 1 -- start start 2024-09-12T01:33:34.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7fd73b7fe640 1 --2- 172.21.15.16:0/2686339517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c15a4e0 0x7fd73c06da90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00f9b0 tx=0x7fd72c02e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb10ba9dc257eacf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f0e3e381640 1 -- 172.21.15.16:0/4159325052 shutdown_connections 2024-09-12T01:33:34.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f0e3e381640 1 --2- 172.21.15.16:0/4159325052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e380af5d0 0x7f0e38159e20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f0e3e381640 1 --2- 172.21.15.16:0/4159325052 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e380ccf80 0x7f0e380af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f0e3e381640 1 -- 172.21.15.16:0/4159325052 >> 172.21.15.16:0/4159325052 conn(0x7f0e380b1af0 msgr2=0x7f0e380b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f68f550a640 1 -- 172.21.15.16:0/2323872039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d8005020 msgr2=0x7f68d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f68f550a640 1 --2- 172.21.15.16:0/2323872039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d8005020 0x7f68d800b9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f68e400aa70 tx=0x7f68d8005830 comp rx=0 tx=0).stop 2024-09-12T01:33:34.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f5c4b427640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5c3c00b770 0x7f5c3c039250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f5c4b427640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5c3c039790 0x7f5c3c054cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f5c4b427640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c3c009c30 con 0x7f5c3c036020 2024-09-12T01:33:34.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f5c4b427640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5c3c006e30 con 0x7f5c3c039790 2024-09-12T01:33:34.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.139+0000 7f5c4b427640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f5c3c007110 con 0x7f5c3c00b770 2024-09-12T01:33:34.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f68ef7fe640 1 -- 172.21.15.16:0/2323872039 reap_dead start 2024-09-12T01:33:34.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f0e3e381640 1 -- 172.21.15.16:0/4159325052 shutdown_connections 2024-09-12T01:33:34.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f0e3e381640 1 -- 172.21.15.16:0/4159325052 wait complete. 2024-09-12T01:33:34.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f8f7759e640 1 --2- 172.21.15.16:0/2216364084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f780af5d0 0x7f8f780ce2c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009d90 tx=0x7f8f7002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb0d730b33fcb77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.138+0000 7fcb3988b640 1 -- start start 2024-09-12T01:33:34.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7fcb3988b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c033800 0x7fcb2c006f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7fcb3988b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb2c0397f0 con 0x7fcb2c039250 2024-09-12T01:33:34.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f5c491c3640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5c3c036020 0x7f5c3c006460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53678/0 (socket says 172.21.15.16:53678) 2024-09-12T01:33:34.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f5c491c3640 1 -- 172.21.15.16:0/1257686700 learned_addr learned my addr 172.21.15.16:0/1257686700 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:34.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f5c499c4640 1 --2- 172.21.15.16:0/1257686700 >> v2:172.21.15.16:3300/0 conn(0x7f5c3c039790 0x7f5c3c054cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f5c489c2640 1 --2- 172.21.15.16:0/1257686700 >> v2:172.21.15.33:3300/0 conn(0x7f5c3c00b770 0x7f5c3c039250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7fd73a7fc640 1 -- 172.21.15.16:0/2686339517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c03b070 con 0x7fd73c15a4e0 2024-09-12T01:33:34.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7fcb32ffd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c033800 0x7fcb2c006f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7fd73a7fc640 1 -- 172.21.15.16:0/2686339517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72c036440 con 0x7fd73c15a4e0 2024-09-12T01:33:34.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7fd73a7fc640 1 -- 172.21.15.16:0/2686339517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734001a10 con 0x7fd73c15a4e0 2024-09-12T01:33:34.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7fcb32ffd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c033800 0x7fcb2c006f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44450/0 (socket says 172.21.15.16:44450) 2024-09-12T01:33:34.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7fcb32ffd640 1 -- 172.21.15.16:0/619390999 learned_addr learned my addr 172.21.15.16:0/619390999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:34.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7fcb3988b640 1 -- 172.21.15.16:0/619390999 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb2c039ad0 con 0x7fcb2c036730 2024-09-12T01:33:34.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7fcb3988b640 1 -- 172.21.15.16:0/619390999 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2c0074d0 con 0x7fcb2c033800 2024-09-12T01:33:34.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f849aa45640 1 -- 172.21.15.16:0/1000656319 shutdown_connections 2024-09-12T01:33:34.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f68f550a640 1 -- 172.21.15.16:0/2323872039 shutdown_connections 2024-09-12T01:33:34.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f68f550a640 1 -- 172.21.15.16:0/2323872039 >> 172.21.15.16:0/2323872039 conn(0x7f68e801ea50 msgr2=0x7f68e8038650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f5c3bfff640 1 -- 172.21.15.16:0/1257686700 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1984444835 0 0) 0x7f5c3c009c30 con 0x7f5c3c036020 2024-09-12T01:33:34.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f5c3bfff640 1 -- 172.21.15.16:0/1257686700 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c30003520 con 0x7f5c3c036020 2024-09-12T01:33:34.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.140+0000 7f849aa45640 1 -- 172.21.15.16:0/1000656319 wait complete. 2024-09-12T01:33:34.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7f68f550a640 1 -- 172.21.15.16:0/2323872039 shutdown_connections 2024-09-12T01:33:34.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7f5c499c4640 1 -- 172.21.15.16:0/1257686700 >> v2:172.21.15.33:3300/0 conn(0x7f5c3c00b770 msgr2=0x7f5c3c039250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:34.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7f5c499c4640 1 --2- 172.21.15.16:0/1257686700 >> v2:172.21.15.33:3300/0 conn(0x7f5c3c00b770 0x7f5c3c039250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7f5c499c4640 1 -- 172.21.15.16:0/1257686700 >> v1:172.21.15.16:6789/0 conn(0x7f5c3c036020 legacy=0x7f5c3c006460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7f5c499c4640 1 -- 172.21.15.16:0/1257686700 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3c009ed0 con 0x7f5c3c039790 2024-09-12T01:33:34.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fd741fbd640 1 -- 172.21.15.16:0/2686339517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c15a4e0 msgr2=0x7fd73c06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fd741fbd640 1 --2- 172.21.15.16:0/2686339517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c15a4e0 0x7fd73c06da90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00f9b0 tx=0x7fd72c02e430 comp rx=0 tx=0).stop 2024-09-12T01:33:34.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7f68f550a640 1 -- 172.21.15.16:0/2323872039 wait complete. 2024-09-12T01:33:34.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7f8f75d9b640 1 -- 172.21.15.16:0/2216364084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70007740 con 0x7f8f780af5d0 2024-09-12T01:33:34.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7f8f75d9b640 1 -- 172.21.15.16:0/2216364084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f700078e0 con 0x7f8f780af5d0 2024-09-12T01:33:34.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7f8f75d9b640 1 -- 172.21.15.16:0/2216364084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70007cb0 con 0x7f8f780af5d0 2024-09-12T01:33:34.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7f8f7d01e640 1 -- 172.21.15.16:0/2216364084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f780af5d0 msgr2=0x7f8f780ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7f8f7d01e640 1 --2- 172.21.15.16:0/2216364084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f780af5d0 0x7f8f780ce2c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009d90 tx=0x7f8f7002a660 comp rx=0 tx=0).stop 2024-09-12T01:33:34.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7f0e3e381640 1 Processor -- start 2024-09-12T01:33:34.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7f8f7d01e640 1 -- 172.21.15.16:0/2216364084 shutdown_connections 2024-09-12T01:33:34.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7f8f7d01e640 1 --2- 172.21.15.16:0/2216364084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f780af5d0 0x7f8f780ce2c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7f8f7d01e640 1 --2- 172.21.15.16:0/2216364084 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8f780ccf80 0x7f8f780af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7f849aa45640 1 Processor -- start 2024-09-12T01:33:34.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7fd741fbd640 1 -- 172.21.15.16:0/2686339517 shutdown_connections 2024-09-12T01:33:34.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb31ffb640 1 -- 172.21.15.16:0/619390999 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1518257612 0 0) 0x7fcb2c039ad0 con 0x7fcb2c036730 2024-09-12T01:33:34.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb31ffb640 1 -- 172.21.15.16:0/619390999 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb1c005730 con 0x7fcb2c036730 2024-09-12T01:33:34.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb31ffb640 1 -- 172.21.15.16:0/619390999 <== mon.1 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 136924963 0 0) 0x7fcb1c005730 con 0x7fcb2c036730 2024-09-12T01:33:34.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7fd741fbd640 1 --2- 172.21.15.16:0/2686339517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c15a4e0 0x7fd73c06da90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7fd741fbd640 1 -- 172.21.15.16:0/2686339517 >> 172.21.15.16:0/2686339517 conn(0x7fd73c0b1bd0 msgr2=0x7fd73c0b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb31ffb640 1 -- 172.21.15.16:0/619390999 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c033800 msgr2=0x7fcb2c006f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:34.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb31ffb640 1 --2- 172.21.15.16:0/619390999 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c033800 0x7fcb2c006f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb31ffb640 1 -- 172.21.15.16:0/619390999 >> v1:172.21.15.16:6789/0 conn(0x7fcb2c039250 legacy=0x7fcb2c036020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb31ffb640 1 -- 172.21.15.16:0/619390999 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c039db0 con 0x7fcb2c036730 2024-09-12T01:33:34.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb32ffd640 1 --2- 172.21.15.16:0/619390999 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c033800 0x7fcb2c006f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:34.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7f0e3e381640 1 -- start start 2024-09-12T01:33:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb31ffb640 1 -- 172.21.15.16:0/619390999 <== mon.1 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7fcb34050800 con 0x7fcb2c036730 2024-09-12T01:33:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb31ffb640 1 -- 172.21.15.16:0/619390999 >> v1:172.21.15.33:6789/0 conn(0x7fcb2c036730 legacy=0x7fcb2c0de1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb31ffb640 1 --2- 172.21.15.16:0/619390999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c005bf0 0x7fcb1c006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb31ffb640 1 --2- 172.21.15.16:0/619390999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1c006550 0x7fcb1c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb32ffd640 1 --2- 172.21.15.16:0/619390999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c005bf0 0x7fcb1c006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb31ffb640 1 --2- 172.21.15.16:0/619390999 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c033800 0x7fcb1c0070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.141+0000 7fcb31ffb640 1 -- 172.21.15.16:0/619390999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c0074d0 con 0x7fcb1c006550 2024-09-12T01:33:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7fcb31ffb640 1 -- 172.21.15.16:0/619390999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c007890 con 0x7fcb1c005bf0 2024-09-12T01:33:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7fcb31ffb640 1 -- 172.21.15.16:0/619390999 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fcb1c009c30 con 0x7fcb2c033800 2024-09-12T01:33:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7fcb337fe640 1 --2- 172.21.15.16:0/619390999 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c033800 0x7fcb1c0070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7fcb32ffd640 1 -- 172.21.15.16:0/619390999 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c033800 msgr2=0x7fcb1c0070c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7fcb32ffd640 1 --2- 172.21.15.16:0/619390999 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c033800 0x7fcb1c0070c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7fcb32ffd640 1 -- 172.21.15.16:0/619390999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1c006550 msgr2=0x7fcb1c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:34.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7fcb32ffd640 1 --2- 172.21.15.16:0/619390999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1c006550 0x7fcb1c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7fcb32ffd640 1 -- 172.21.15.16:0/619390999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c005730 con 0x7fcb1c005bf0 2024-09-12T01:33:34.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.142+0000 7f8f7d01e640 1 -- 172.21.15.16:0/2216364084 >> 172.21.15.16:0/2216364084 conn(0x7f8f780b1af0 msgr2=0x7f8f780b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.143+0000 7f68f550a640 1 Processor -- start 2024-09-12T01:33:34.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.143+0000 7f8f7d01e640 1 -- 172.21.15.16:0/2216364084 shutdown_connections 2024-09-12T01:33:34.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.143+0000 7fd741fbd640 1 -- 172.21.15.16:0/2686339517 shutdown_connections 2024-09-12T01:33:34.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.143+0000 7f849aa45640 1 -- start start 2024-09-12T01:33:34.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.143+0000 7f8f7d01e640 1 -- 172.21.15.16:0/2216364084 wait complete. 2024-09-12T01:33:34.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.143+0000 7f68f550a640 1 -- start start 2024-09-12T01:33:34.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f5c499c4640 1 --2- 172.21.15.16:0/1257686700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c3c039790 0x7f5c3c054cb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5c44000b60 tx=0x7f5c440951a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9166224dcb16353c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f68f550a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68d8005020 0x7f68e80213a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f68f550a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e804bbd0 0x7f68e80ea130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f68f550a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68e80ea670 0x7f68e8049070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f68f550a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68e8034b50 con 0x7f68e804bbd0 2024-09-12T01:33:34.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f68f550a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68e800b550 con 0x7f68d8005020 2024-09-12T01:33:34.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f68f550a640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f68e800c460 con 0x7f68e80ea670 2024-09-12T01:33:34.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f849aa45640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84940c5670 0x7f84940ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f849aa45640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8494166cb0 0x7f8494151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f849aa45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f849414e840 0x7f849414d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f849aa45640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f849406d4c0 con 0x7f849414e840 2024-09-12T01:33:34.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f849aa45640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f849406d340 con 0x7f84940c5670 2024-09-12T01:33:34.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f849aa45640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f84940d0620 con 0x7f8494166cb0 2024-09-12T01:33:34.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.143+0000 7fd741fbd640 1 -- 172.21.15.16:0/2686339517 wait complete. 2024-09-12T01:33:34.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7fcb32ffd640 1 --2- 172.21.15.16:0/619390999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c005bf0 0x7fcb1c006010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400f550 tx=0x7fcb1c007a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d339023ebaa44c5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f68ef7fe640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68e80ea670 0x7f68e8049070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f68ef7fe640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68e80ea670 0x7f68e8049070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44462/0 (socket says 172.21.15.16:44462) 2024-09-12T01:33:34.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f68ef7fe640 1 -- 172.21.15.16:0/3410298768 learned_addr learned my addr 172.21.15.16:0/3410298768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:34.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f0e3e381640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e380af5d0 0x7f0e38153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f0e3e381640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e380ccf80 0x7f0e38154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f8493fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84940c5670 0x7f84940ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f8493fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84940c5670 0x7f84940ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45608/0 (socket says 172.21.15.16:45608) 2024-09-12T01:33:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f8498fe2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f849414e840 0x7f849414d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f8493fff640 1 -- 172.21.15.16:0/4032448592 learned_addr learned my addr 172.21.15.16:0/4032448592 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7fcb31ffb640 1 -- 172.21.15.16:0/619390999 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24058070 con 0x7fcb1c005bf0 2024-09-12T01:33:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f0e3e381640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e380cd360 0x7f0e380c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7fcb31ffb640 1 -- 172.21.15.16:0/619390999 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb24053600 con 0x7fcb1c005bf0 2024-09-12T01:33:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f68ee7fc640 1 --2- 172.21.15.16:0/3410298768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e804bbd0 0x7f68e80ea130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f68ef7fe640 1 -- 172.21.15.16:0/3410298768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68d8005020 msgr2=0x7f68e80213a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f68ef7fe640 1 --2- 172.21.15.16:0/3410298768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68d8005020 0x7f68e80213a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f68ef7fe640 1 -- 172.21.15.16:0/3410298768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e804bbd0 msgr2=0x7f68e80ea130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f68ef7fe640 1 --2- 172.21.15.16:0/3410298768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e804bbd0 0x7f68e80ea130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f68ef7fe640 1 -- 172.21.15.16:0/3410298768 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e8048460 con 0x7f68e80ea670 2024-09-12T01:33:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f0e37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e380af5d0 0x7f0e38153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f0e37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e380af5d0 0x7f0e38153ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42302/0 (socket says 172.21.15.16:42302) 2024-09-12T01:33:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f0e377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e380ccf80 0x7f0e38154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f0e377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e380ccf80 0x7f0e38154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45640/0 (socket says 172.21.15.16:45640) 2024-09-12T01:33:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f0e3c91e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e380cd360 0x7f0e380c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f0e3c91e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e380cd360 0x7f0e380c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44478/0 (socket says 172.21.15.16:44478) 2024-09-12T01:33:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f0e37fff640 1 -- 172.21.15.16:0/4215659587 learned_addr learned my addr 172.21.15.16:0/4215659587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f0e3e381640 1 -- 172.21.15.16:0/4215659587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e380cf2d0 con 0x7f0e380af5d0 2024-09-12T01:33:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f0e3e381640 1 -- 172.21.15.16:0/4215659587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3815def0 con 0x7f0e380ccf80 2024-09-12T01:33:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f0e3e381640 1 -- 172.21.15.16:0/4215659587 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0e380cf450 con 0x7f0e380cd360 2024-09-12T01:33:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7fd741fbd640 1 Processor -- start 2024-09-12T01:33:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f5c3bfff640 1 -- 172.21.15.16:0/1257686700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c440a2070 con 0x7f5c3c039790 2024-09-12T01:33:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.144+0000 7f5c3bfff640 1 -- 172.21.15.16:0/1257686700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4409d440 con 0x7f5c3c039790 2024-09-12T01:33:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f8493fff640 1 -- 172.21.15.16:0/4032448592 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8494166cb0 msgr2=0x7f8494151820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T01:33:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f8493fff640 1 --2- 172.21.15.16:0/4032448592 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8494166cb0 0x7f8494151820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f8493fff640 1 -- 172.21.15.16:0/4032448592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f849414e840 msgr2=0x7f849414d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f8493fff640 1 --2- 172.21.15.16:0/4032448592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f849414e840 0x7f849414d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f8493fff640 1 -- 172.21.15.16:0/4032448592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8494070d10 con 0x7f84940c5670 2024-09-12T01:33:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7fcb3988b640 1 -- 172.21.15.16:0/619390999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c005bf0 msgr2=0x7fcb1c006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7fcb3988b640 1 --2- 172.21.15.16:0/619390999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c005bf0 0x7fcb1c006010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400f550 tx=0x7fcb1c007a40 comp rx=0 tx=0).stop 2024-09-12T01:33:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7fcb337fe640 1 -- 172.21.15.16:0/619390999 reap_dead start 2024-09-12T01:33:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f68ef7fe640 1 --2- 172.21.15.16:0/3410298768 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68e80ea670 0x7f68e8049070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f68e400a120 tx=0x7f68e4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f5c4b427640 1 -- 172.21.15.16:0/1257686700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c3c039790 msgr2=0x7f5c3c054cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f5c4b427640 1 --2- 172.21.15.16:0/1257686700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c3c039790 0x7f5c3c054cb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5c44000b60 tx=0x7f5c440951a0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7fcb3988b640 1 -- 172.21.15.16:0/619390999 shutdown_connections 2024-09-12T01:33:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7fcb3988b640 1 -- 172.21.15.16:0/619390999 >> 172.21.15.16:0/619390999 conn(0x7fcb2c01d930 msgr2=0x7fcb2c0545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f8f7d01e640 1 Processor -- start 2024-09-12T01:33:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f8f7d01e640 1 -- start start 2024-09-12T01:33:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f68d77fe640 1 -- 172.21.15.16:0/3410298768 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4041030 con 0x7f68e80ea670 2024-09-12T01:33:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f0e37fff640 1 -- 172.21.15.16:0/4215659587 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e380cd360 msgr2=0x7f0e380c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f0e37fff640 1 --2- 172.21.15.16:0/4215659587 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e380cd360 0x7f0e380c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f0e37fff640 1 -- 172.21.15.16:0/4215659587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e380ccf80 msgr2=0x7f0e38154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f0e37fff640 1 --2- 172.21.15.16:0/4215659587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e380ccf80 0x7f0e38154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f0e37fff640 1 -- 172.21.15.16:0/4215659587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e380c9450 con 0x7f0e380af5d0 2024-09-12T01:33:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f5c4b427640 1 -- 172.21.15.16:0/1257686700 shutdown_connections 2024-09-12T01:33:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f5c4b427640 1 --2- 172.21.15.16:0/1257686700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c3c039790 0x7f5c3c054cb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f5c4b427640 1 --2- 172.21.15.16:0/1257686700 >> v2:172.21.15.33:3300/0 conn(0x7f5c3c00b770 0x7f5c3c039250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f5c4b427640 1 -- 172.21.15.16:0/1257686700 >> 172.21.15.16:0/1257686700 conn(0x7f5c3c01d930 msgr2=0x7f5c3c0545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f5c4b427640 1 -- 172.21.15.16:0/1257686700 shutdown_connections 2024-09-12T01:33:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f5c4b427640 1 -- 172.21.15.16:0/1257686700 wait complete. 2024-09-12T01:33:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7f8493fff640 1 --2- 172.21.15.16:0/4032448592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84940c5670 0x7f84940ca5c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84880098c0 tx=0x7f84880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f68f550a640 1 -- 172.21.15.16:0/3410298768 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e8021aa0 con 0x7f68e80ea670 2024-09-12T01:33:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f68f550a640 1 -- 172.21.15.16:0/3410298768 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e801f3c0 con 0x7f68e80ea670 2024-09-12T01:33:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f8490ff9640 1 -- 172.21.15.16:0/4032448592 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488042020 con 0x7f84940c5670 2024-09-12T01:33:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f849aa45640 1 -- 172.21.15.16:0/4032448592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8494070ff0 con 0x7f84940c5670 2024-09-12T01:33:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f849aa45640 1 -- 172.21.15.16:0/4032448592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8494165a60 con 0x7f84940c5670 2024-09-12T01:33:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f0e37fff640 1 --2- 172.21.15.16:0/4215659587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e380af5d0 0x7f0e38153ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009ed0 tx=0x7f0e2c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f8490ff9640 1 -- 172.21.15.16:0/4032448592 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848803d670 con 0x7f84940c5670 2024-09-12T01:33:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f8490ff9640 1 -- 172.21.15.16:0/4032448592 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488037040 con 0x7f84940c5670 2024-09-12T01:33:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.145+0000 7fd741fbd640 1 -- start start 2024-09-12T01:33:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7fd741fbd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c0cd4a0 0x7fd73c0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f8f7d01e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f780af5d0 0x7f8f781532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f7d01e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f780b25e0 0x7f8f780c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f0e34ff9640 1 -- 172.21.15.16:0/4215659587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c016b80 con 0x7f0e380af5d0 2024-09-12T01:33:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7fd741fbd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c0ce580 0x7fd73c1e4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7fd741fbd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd73c15a4e0 0x7fd73c1e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7fd73b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c0cd4a0 0x7fd73c0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7fd73b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c0cd4a0 0x7fd73c0b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45656/0 (socket says 172.21.15.16:45656) 2024-09-12T01:33:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f7d01e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8f780ccf80 0x7f8f781515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f76d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f780af5d0 0x7f8f781532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f7d01e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7815a530 con 0x7f8f780b25e0 2024-09-12T01:33:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f7d01e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7815def0 con 0x7f8f780af5d0 2024-09-12T01:33:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f76d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f780af5d0 0x7f8f781532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45672/0 (socket says 172.21.15.16:45672) 2024-09-12T01:33:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f7d01e640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7815a6b0 con 0x7f8f780ccf80 2024-09-12T01:33:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f76d9d640 1 -- 172.21.15.16:0/1146608636 learned_addr learned my addr 172.21.15.16:0/1146608636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f7759e640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8f780ccf80 0x7f8f781515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7fcb3988b640 1 -- 172.21.15.16:0/619390999 shutdown_connections 2024-09-12T01:33:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f0e34ff9640 1 -- 172.21.15.16:0/4215659587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e2c016d20 con 0x7f0e380af5d0 2024-09-12T01:33:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f0e34ff9640 1 -- 172.21.15.16:0/4215659587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c00ca50 con 0x7f0e380af5d0 2024-09-12T01:33:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f0e3e381640 1 -- 172.21.15.16:0/4215659587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e380c9730 con 0x7f0e380af5d0 2024-09-12T01:33:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f5c4b427640 1 Processor -- start 2024-09-12T01:33:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f5c4b427640 1 -- start start 2024-09-12T01:33:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f76d9d640 1 -- 172.21.15.16:0/1146608636 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8f780ccf80 msgr2=0x7f8f781515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f76d9d640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8f780ccf80 0x7f8f781515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f76d9d640 1 -- 172.21.15.16:0/1146608636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f780b25e0 msgr2=0x7f8f780c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f76d9d640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f780b25e0 0x7f8f780c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f76d9d640 1 -- 172.21.15.16:0/1146608636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f780c3af0 con 0x7f8f780af5d0 2024-09-12T01:33:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f8f7759e640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8f780ccf80 0x7f8f781515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f0e3e381640 1 -- 172.21.15.16:0/4215659587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3809aeb0 con 0x7f0e380af5d0 2024-09-12T01:33:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e377fe640 1 --2- 172.21.15.16:0/4215659587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e380ccf80 0x7f0e38154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7fd741fbd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c0ce9a0 con 0x7fd73c0ce580 2024-09-12T01:33:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7fd741fbd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c06e730 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7fd741fbd640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fd73c0b26c0 con 0x7fd73c15a4e0 2024-09-12T01:33:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7fd73affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c0ce580 0x7fd73c1e4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7fd73b7fe640 1 -- 172.21.15.16:0/2101952030 learned_addr learned my addr 172.21.15.16:0/2101952030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7fd73bfff640 1 --2- 172.21.15.16:0/2101952030 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd73c15a4e0 0x7fd73c1e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f0e6ecaf640 1 Processor -- start 2024-09-12T01:33:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.146+0000 7f0e6ecaf640 1 -- start start 2024-09-12T01:33:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e6ecaf640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e600dec70 0x7f0e60006550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e6ecaf640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e60009af0 0x7f0e60009f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e6ecaf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e6001c5c0 con 0x7f0e600323c0 2024-09-12T01:33:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e6ecaf640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f0e6000a480 con 0x7f0e600dec70 2024-09-12T01:33:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e6ecaf640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0e6001c830 con 0x7f0e60009af0 2024-09-12T01:33:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7fd73b7fe640 1 -- 172.21.15.16:0/2101952030 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd73c15a4e0 msgr2=0x7fd73c1e4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7fd73b7fe640 1 --2- 172.21.15.16:0/2101952030 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd73c15a4e0 0x7fd73c1e4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7fd73b7fe640 1 -- 172.21.15.16:0/2101952030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c0ce580 msgr2=0x7fd73c1e4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7fd73b7fe640 1 --2- 172.21.15.16:0/2101952030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c0ce580 0x7fd73c1e4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7fd73b7fe640 1 -- 172.21.15.16:0/2101952030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c1565f0 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e6dcad640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e600dec70 0x7f0e60006550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e6dcad640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e600dec70 0x7f0e60006550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45674/0 (socket says 172.21.15.16:45674) 2024-09-12T01:33:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e6dcad640 1 -- 172.21.15.16:0/297179342 learned_addr learned my addr 172.21.15.16:0/297179342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e6e4ae640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e60009af0 0x7f0e60009f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7fcb3988b640 1 -- 172.21.15.16:0/619390999 wait complete. 2024-09-12T01:33:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f68d77fe640 1 -- 172.21.15.16:0/3410298768 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68e402aab0 con 0x7f68e80ea670 2024-09-12T01:33:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.147+0000 7f68d77fe640 1 -- 172.21.15.16:0/3410298768 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4041030 con 0x7f68e80ea670 2024-09-12T01:33:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e6dcad640 1 -- 172.21.15.16:0/297179342 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e60009af0 msgr2=0x7f0e60009f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e6dcad640 1 --2- 172.21.15.16:0/297179342 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e60009af0 0x7f0e60009f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e6dcad640 1 -- 172.21.15.16:0/297179342 >> v1:172.21.15.16:6789/0 conn(0x7f0e600323c0 legacy=0x7f0e60052350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.148+0000 7f0e6dcad640 1 -- 172.21.15.16:0/297179342 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6001cb10 con 0x7f0e600dec70 2024-09-12T01:33:34.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f68d57fa640 1 -- 172.21.15.16:0/3410298768 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e8034690 con 0x7f68e80ea670 2024-09-12T01:33:34.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c4b427640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c3c00b770 0x7f5c3c005330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f8490ff9640 1 -- 172.21.15.16:0/4032448592 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8488045020 con 0x7f84940c5670 2024-09-12T01:33:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c4b427640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c3c036020 0x7f5c3c0f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c4b427640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c3c039790 0x7f5c3c0f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c491c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c3c00b770 0x7f5c3c005330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7fd73b7fe640 1 --2- 172.21.15.16:0/2101952030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c0cd4a0 0x7fd73c0b3b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00f9b0 tx=0x7fd72c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c4b427640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c0551f0 con 0x7f5c3c00b770 2024-09-12T01:33:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c491c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c3c00b770 0x7f5c3c005330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42310/0 (socket says 172.21.15.16:42310) 2024-09-12T01:33:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c491c3640 1 -- 172.21.15.16:0/2642019771 learned_addr learned my addr 172.21.15.16:0/2642019771 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c4b427640 1 -- 172.21.15.16:0/2642019771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c0073f0 con 0x7f5c3c036020 2024-09-12T01:33:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c4b427640 1 -- 172.21.15.16:0/2642019771 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f5c3c055370 con 0x7f5c3c039790 2024-09-12T01:33:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f8f76d9d640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f780af5d0 0x7f8f781532c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800e870 tx=0x7f8f6800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7fcb3988b640 1 Processor -- start 2024-09-12T01:33:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c491c3640 1 -- 172.21.15.16:0/2642019771 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c3c039790 msgr2=0x7f5c3c0f61a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T01:33:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c491c3640 1 --2- 172.21.15.16:0/2642019771 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c3c039790 0x7f5c3c0f61a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7fd723fff640 1 -- 172.21.15.16:0/2101952030 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c044070 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c491c3640 1 -- 172.21.15.16:0/2642019771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c3c036020 msgr2=0x7f5c3c0f5c60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T01:33:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c491c3640 1 --2- 172.21.15.16:0/2642019771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c3c036020 0x7f5c3c0f5c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.149+0000 7f5c491c3640 1 -- 172.21.15.16:0/2642019771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3c020e60 con 0x7f5c3c00b770 2024-09-12T01:33:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7f8490ff9640 1 --2- 172.21.15.16:0/4032448592 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f847806bf30 0x7f847806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7fd741fbd640 1 -- 172.21.15.16:0/2101952030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c09ab60 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7f8490ff9640 1 -- 172.21.15.16:0/4032448592 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f84880ab7b0 con 0x7f84940c5670 2024-09-12T01:33:34.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7fd741fbd640 1 -- 172.21.15.16:0/2101952030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c09b120 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7fd723fff640 1 -- 172.21.15.16:0/2101952030 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72c004030 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7f0e6dcad640 1 --2- 172.21.15.16:0/297179342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e600dec70 0x7f0e60006550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400c3c0 tx=0x7f0e64029f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bebb6ff30c7ac7b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7f5c491c3640 1 --2- 172.21.15.16:0/2642019771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c3c00b770 0x7f5c3c005330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c480 tx=0x7f5c4000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7fd723fff640 1 -- 172.21.15.16:0/2101952030 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c03b070 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7f84937fe640 1 --2- 172.21.15.16:0/4032448592 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f847806bf30 0x7f847806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.151+0000 7f84937fe640 1 --2- 172.21.15.16:0/4032448592 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f847806bf30 0x7f847806e3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8480005880 tx=0x7f8480007a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7f68d77fe640 1 -- 172.21.15.16:0/3410298768 <== mon.2 v2:172.21.15.33:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f68e4044070 con 0x7f68e80ea670 2024-09-12T01:33:34.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7f68d77fe640 1 --2- 172.21.15.16:0/3410298768 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f68dc06c060 0x7f68dc06e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.151+0000 7f0e34ff9640 1 -- 172.21.15.16:0/4215659587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e2c00cbf0 con 0x7f0e380af5d0 2024-09-12T01:33:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7f0e34ff9640 1 --2- 172.21.15.16:0/4215659587 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e2406bf30 0x7f0e2406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7f0e34ff9640 1 -- 172.21.15.16:0/4215659587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0e2c08f860 con 0x7f0e380af5d0 2024-09-12T01:33:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7f8f5f7fe640 1 -- 172.21.15.16:0/1146608636 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68017070 con 0x7f8f780af5d0 2024-09-12T01:33:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7f8f7d01e640 1 -- 172.21.15.16:0/1146608636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f780c3d90 con 0x7f8f780af5d0 2024-09-12T01:33:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7f8f7d01e640 1 -- 172.21.15.16:0/1146608636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f780c94a0 con 0x7f8f780af5d0 2024-09-12T01:33:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7f8f5f7fe640 1 -- 172.21.15.16:0/1146608636 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6800c940 con 0x7f8f780af5d0 2024-09-12T01:33:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.151+0000 7fd741fbd640 1 -- 172.21.15.16:0/2101952030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73c06da90 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7fcb3988b640 1 -- start start 2024-09-12T01:33:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.151+0000 7fcb3988b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb2c04df90 0x7fcb2c178440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.151+0000 7fcb3988b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb2c0c74b0 0x7fcb2c0c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.151+0000 7fcb3988b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c031950 0x7fcb2c031dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.151+0000 7fcb3988b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c033be0 con 0x7fcb2c0c74b0 2024-09-12T01:33:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.151+0000 7fcb3988b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c03a400 con 0x7fcb2c04df90 2024-09-12T01:33:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.151+0000 7fcb3988b640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2c054cb0 con 0x7fcb2c031950 2024-09-12T01:33:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7fcb32ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb2c04df90 0x7fcb2c178440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7fcb32ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb2c04df90 0x7fcb2c178440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45694/0 (socket says 172.21.15.16:45694) 2024-09-12T01:33:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7fcb32ffd640 1 -- 172.21.15.16:0/1754262119 learned_addr learned my addr 172.21.15.16:0/1754262119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.151+0000 7f0e6ccab640 1 -- 172.21.15.16:0/297179342 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e64008df0 con 0x7f0e600dec70 2024-09-12T01:33:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.151+0000 7f0e6ccab640 1 -- 172.21.15.16:0/297179342 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e64004030 con 0x7f0e600dec70 2024-09-12T01:33:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.151+0000 7f0e6ccab640 1 -- 172.21.15.16:0/297179342 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e64004460 con 0x7f0e600dec70 2024-09-12T01:33:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/297179342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e600dec70 msgr2=0x7f0e60006550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7f0e6ecaf640 1 --2- 172.21.15.16:0/297179342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e600dec70 0x7f0e60006550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400c3c0 tx=0x7f0e64029f90 comp rx=0 tx=0).stop 2024-09-12T01:33:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7f5c39ffb640 1 -- 172.21.15.16:0/2642019771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40017070 con 0x7f5c3c00b770 2024-09-12T01:33:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7f5c39ffb640 1 -- 172.21.15.16:0/2642019771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c40005ce0 con 0x7f5c3c00b770 2024-09-12T01:33:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7f5c39ffb640 1 -- 172.21.15.16:0/2642019771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40007270 con 0x7f5c3c00b770 2024-09-12T01:33:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.150+0000 7f5c4b427640 1 -- 172.21.15.16:0/2642019771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3c021100 con 0x7f5c3c00b770 2024-09-12T01:33:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.151+0000 7f5c4b427640 1 -- 172.21.15.16:0/2642019771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c3c0216c0 con 0x7f5c3c00b770 2024-09-12T01:33:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7fd723fff640 1 -- 172.21.15.16:0/2101952030 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd72c047070 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7f68eeffd640 1 --2- 172.21.15.16:0/3410298768 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f68dc06c060 0x7f68dc06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7f5c4b427640 1 -- 172.21.15.16:0/2642019771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c3c039250 con 0x7f5c3c00b770 2024-09-12T01:33:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/297179342 shutdown_connections 2024-09-12T01:33:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7f0e6ecaf640 1 --2- 172.21.15.16:0/297179342 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e60009af0 0x7f0e60009f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7f0e6ecaf640 1 --2- 172.21.15.16:0/297179342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e600dec70 0x7f0e60006550 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/297179342 >> 172.21.15.16:0/297179342 conn(0x7f0e60036090 msgr2=0x7f0e600364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.152+0000 7f0e3e381640 1 -- 172.21.15.16:0/4215659587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e380af090 con 0x7f0e380af5d0 2024-09-12T01:33:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7f0e377fe640 1 --2- 172.21.15.16:0/4215659587 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e2406bf30 0x7f0e2406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7f849aa45640 1 -- 172.21.15.16:0/4032448592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8474005290 con 0x7f84940c5670 2024-09-12T01:33:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7fcb327fc640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb2c0c74b0 0x7fcb2c0c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7fcb337fe640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c031950 0x7fcb2c031dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7fcb32ffd640 1 -- 172.21.15.16:0/1754262119 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c031950 msgr2=0x7fcb2c031dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7f68d77fe640 1 -- 172.21.15.16:0/3410298768 <== mon.2 v2:172.21.15.33:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f68e4038040 con 0x7f68e80ea670 2024-09-12T01:33:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7fcb32ffd640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c031950 0x7fcb2c031dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7f0e377fe640 1 --2- 172.21.15.16:0/4215659587 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e2406bf30 0x7f0e2406e3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0e28034000 tx=0x7f0e2802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.154+0000 7f68eeffd640 1 --2- 172.21.15.16:0/3410298768 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f68dc06c060 0x7f68dc06e520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f68e0006a30 tx=0x7f68e00069c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.154+0000 7fd723fff640 1 --2- 172.21.15.16:0/2101952030 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd72806bf00 0x7fd72806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7fcb32ffd640 1 -- 172.21.15.16:0/1754262119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb2c0c74b0 msgr2=0x7fcb2c0c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7fcb32ffd640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb2c0c74b0 0x7fcb2c0c5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7fcb32ffd640 1 -- 172.21.15.16:0/1754262119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c0c7c50 con 0x7fcb2c04df90 2024-09-12T01:33:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7fcb32ffd640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb2c04df90 0x7fcb2c178440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400fb30 tx=0x7fcb24009210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7fcb337fe640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c031950 0x7fcb2c031dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.154+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/297179342 shutdown_connections 2024-09-12T01:33:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.154+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/297179342 wait complete. 2024-09-12T01:33:34.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.154+0000 7fd723fff640 1 -- 172.21.15.16:0/2101952030 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd72c0aab60 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.154+0000 7fd73affd640 1 --2- 172.21.15.16:0/2101952030 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd72806bf00 0x7fd72806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.153+0000 7f8f5f7fe640 1 -- 172.21.15.16:0/1146608636 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6800cd30 con 0x7f8f780af5d0 2024-09-12T01:33:34.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.154+0000 7f8f5f7fe640 1 -- 172.21.15.16:0/1146608636 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f6801a070 con 0x7f8f780af5d0 2024-09-12T01:33:34.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.154+0000 7fd73affd640 1 --2- 172.21.15.16:0/2101952030 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd72806bf00 0x7fd72806e3c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd730007fd0 tx=0x7fd730009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.155+0000 7f8f7d01e640 1 -- 172.21.15.16:0/1146608636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f58005290 con 0x7f8f780af5d0 2024-09-12T01:33:34.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.155+0000 7f8f5f7fe640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8f6406be80 0x7f8f6406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.155+0000 7f5c39ffb640 1 -- 172.21.15.16:0/2642019771 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c4001a070 con 0x7f5c3c00b770 2024-09-12T01:33:34.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.155+0000 7f8f5f7fe640 1 -- 172.21.15.16:0/1146608636 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8f6808aea0 con 0x7f8f780af5d0 2024-09-12T01:33:34.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.155+0000 7f0e6ecaf640 1 Processor -- start 2024-09-12T01:33:34.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.155+0000 7f8f7659c640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8f6406be80 0x7f8f6406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.155+0000 7f0e6ecaf640 1 -- start start 2024-09-12T01:33:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.155+0000 7f5c39ffb640 1 --2- 172.21.15.16:0/2642019771 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5c2806be60 0x7f5c2806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.156+0000 7f5c39ffb640 1 -- 172.21.15.16:0/2642019771 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5c4008a8f0 con 0x7f5c3c00b770 2024-09-12T01:33:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.156+0000 7f5c489c2640 1 --2- 172.21.15.16:0/2642019771 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5c2806be60 0x7f5c2806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.156+0000 7f8f7659c640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8f6406be80 0x7f8f6406e340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f780cf6f0 tx=0x7f8f6c007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.156+0000 7f5c489c2640 1 --2- 172.21.15.16:0/2642019771 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5c2806be60 0x7f5c2806e320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c004f30 tx=0x7f5c34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.156+0000 7f68d77fe640 1 -- 172.21.15.16:0/3410298768 <== mon.2 v2:172.21.15.33:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f68e8048460 con 0x7f68e80ea670 2024-09-12T01:33:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.156+0000 7f0e6ecaf640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e60009af0 0x7f0e6003a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.156+0000 7f0e6ecaf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600323c0 0x7f0e600373d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.156+0000 7f0e6ecaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e600dec70 0x7f0e6004db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.156+0000 7f0e6ecaf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e600df050 con 0x7f0e600323c0 2024-09-12T01:33:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.156+0000 7f0e6ecaf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60006a90 con 0x7f0e600dec70 2024-09-12T01:33:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.156+0000 7f0e6ecaf640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0e60032980 con 0x7f0e60009af0 2024-09-12T01:33:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e6dcad640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e60009af0 0x7f0e6003a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e6dcad640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e60009af0 0x7f0e6003a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44534/0 (socket says 172.21.15.16:44534) 2024-09-12T01:33:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e6dcad640 1 -- 172.21.15.16:0/3705770502 learned_addr learned my addr 172.21.15.16:0/3705770502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e6e4ae640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e600dec70 0x7f0e6004db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e6dcad640 1 -- 172.21.15.16:0/3705770502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e600dec70 msgr2=0x7f0e6004db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e6dcad640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e600dec70 0x7f0e6004db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e6dcad640 1 -- 172.21.15.16:0/3705770502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600323c0 msgr2=0x7f0e600373d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e6dcad640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600323c0 0x7f0e600373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e6dcad640 1 -- 172.21.15.16:0/3705770502 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e60054110 con 0x7f0e60009af0 2024-09-12T01:33:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e6e4ae640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e600dec70 0x7f0e6004db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e6dcad640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e60009af0 0x7f0e6003a060 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400c3f0 tx=0x7f0e640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e5e7fc640 1 -- 172.21.15.16:0/3705770502 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e64040020 con 0x7f0e60009af0 2024-09-12T01:33:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e5e7fc640 1 -- 172.21.15.16:0/3705770502 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6400a2e0 con 0x7f0e60009af0 2024-09-12T01:33:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/3705770502 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e60037a60 con 0x7f0e60009af0 2024-09-12T01:33:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.157+0000 7f0e5e7fc640 1 -- 172.21.15.16:0/3705770502 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6400a6d0 con 0x7f0e60009af0 2024-09-12T01:33:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.158+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/3705770502 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e6004b710 con 0x7f0e60009af0 2024-09-12T01:33:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.158+0000 7fcb1bfff640 1 -- 172.21.15.16:0/1754262119 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24063030 con 0x7fcb2c04df90 2024-09-12T01:33:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.158+0000 7fcb3988b640 1 -- 172.21.15.16:0/1754262119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2c0c7f30 con 0x7fcb2c04df90 2024-09-12T01:33:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.158+0000 7fcb3988b640 1 -- 172.21.15.16:0/1754262119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2c0c7930 con 0x7fcb2c04df90 2024-09-12T01:33:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.159+0000 7fd723fff640 1 -- 172.21.15.16:0/2101952030 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd73c1565f0 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.159+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/3705770502 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e6004bb50 con 0x7f0e60009af0 2024-09-12T01:33:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.159+0000 7fcb3988b640 1 -- 172.21.15.16:0/1754262119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2c006f90 con 0x7fcb2c04df90 2024-09-12T01:33:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.160+0000 7f0e5e7fc640 1 -- 172.21.15.16:0/3705770502 <== mon.2 v2:172.21.15.33:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e64043020 con 0x7f0e60009af0 2024-09-12T01:33:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.160+0000 7f5c39ffb640 1 -- 172.21.15.16:0/2642019771 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5c40021020 con 0x7f5c3c00b770 2024-09-12T01:33:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.160+0000 7f0e5e7fc640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e4c06be30 0x7f0e4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.160+0000 7f0e5e7fc640 1 -- 172.21.15.16:0/3705770502 <== mon.2 v2:172.21.15.33:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0e640ab6b0 con 0x7f0e60009af0 2024-09-12T01:33:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.160+0000 7f0e6d4ac640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e4c06be30 0x7f0e4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.161+0000 7f0e6d4ac640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e4c06be30 0x7f0e4c06e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e6004e4b0 tx=0x7f0e58007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.163+0000 7f8490ff9640 1 -- 172.21.15.16:0/4032448592 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f848804c030 con 0x7f84940c5670 2024-09-12T01:33:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.165+0000 7f0e34ff9640 1 -- 172.21.15.16:0/4215659587 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0e2c01a030 con 0x7f0e380af5d0 2024-09-12T01:33:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.165+0000 7f0e5e7fc640 1 -- 172.21.15.16:0/3705770502 <== mon.2 v2:172.21.15.33:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0e6404a030 con 0x7f0e60009af0 2024-09-12T01:33:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.169+0000 7f8f5f7fe640 1 -- 172.21.15.16:0/1146608636 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8f68058f30 con 0x7f8f780af5d0 2024-09-12T01:33:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.168+0000 7fcb1bfff640 1 -- 172.21.15.16:0/1754262119 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb24059040 con 0x7fcb2c04df90 2024-09-12T01:33:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.169+0000 7fcb1bfff640 1 -- 172.21.15.16:0/1754262119 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24058070 con 0x7fcb2c04df90 2024-09-12T01:33:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.170+0000 7fcb1bfff640 1 -- 172.21.15.16:0/1754262119 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb24066070 con 0x7fcb2c04df90 2024-09-12T01:33:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.171+0000 7fcb1bfff640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fcb2006be30 0x7fcb2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.171+0000 7fcb327fc640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fcb2006be30 0x7fcb2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.172+0000 7fcb327fc640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fcb2006be30 0x7fcb2006e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0df0d0 tx=0x7fcb28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.173+0000 7fcb1bfff640 1 -- 172.21.15.16:0/1754262119 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fcb240cdad0 con 0x7fcb2c04df90 2024-09-12T01:33:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.173+0000 7fcb1bfff640 1 -- 172.21.15.16:0/1754262119 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcb2406d070 con 0x7fcb2c04df90 2024-09-12T01:33:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.562+0000 7fd741fbd640 1 -- 172.21.15.16:0/2101952030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd73c0c50f0 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.563+0000 7fd723fff640 1 -- 172.21.15.16:0/2101952030 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd72c078d10 con 0x7fd73c0cd4a0 2024-09-12T01:33:34.562 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2024-09-12T01:33:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.566+0000 7fd721ffb640 1 -- 172.21.15.16:0/2101952030 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd72806bf00 msgr2=0x7fd72806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.566+0000 7fd721ffb640 1 --2- 172.21.15.16:0/2101952030 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd72806bf00 0x7fd72806e3c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd730007fd0 tx=0x7fd730009040 comp rx=0 tx=0).stop 2024-09-12T01:33:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.567+0000 7fd721ffb640 1 -- 172.21.15.16:0/2101952030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c0cd4a0 msgr2=0x7fd73c0b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.567+0000 7fd721ffb640 1 --2- 172.21.15.16:0/2101952030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c0cd4a0 0x7fd73c0b3b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00f9b0 tx=0x7fd72c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.567+0000 7fd721ffb640 1 -- 172.21.15.16:0/2101952030 shutdown_connections 2024-09-12T01:33:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.567+0000 7fd721ffb640 1 --2- 172.21.15.16:0/2101952030 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd72806bf00 0x7fd72806e3c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.567+0000 7fd721ffb640 1 --2- 172.21.15.16:0/2101952030 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd73c15a4e0 0x7fd73c1e4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.567+0000 7fd721ffb640 1 --2- 172.21.15.16:0/2101952030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c0ce580 0x7fd73c1e4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.568+0000 7fd721ffb640 1 --2- 172.21.15.16:0/2101952030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c0cd4a0 0x7fd73c0b3b50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.568+0000 7fd721ffb640 1 -- 172.21.15.16:0/2101952030 >> 172.21.15.16:0/2101952030 conn(0x7fd73c0b1bd0 msgr2=0x7fd73c1e6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.568+0000 7fd721ffb640 1 -- 172.21.15.16:0/2101952030 shutdown_connections 2024-09-12T01:33:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.568+0000 7fd721ffb640 1 -- 172.21.15.16:0/2101952030 wait complete. 2024-09-12T01:33:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.569+0000 7f0e3e381640 1 -- 172.21.15.16:0/4215659587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0e380c9900 con 0x7f0e380af5d0 2024-09-12T01:33:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.570+0000 7f0e34ff9640 1 -- 172.21.15.16:0/4215659587 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e2c05df40 con 0x7f0e380af5d0 2024-09-12T01:33:34.569 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2024-09-12T01:33:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.573+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/3705770502 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0e600321a0 con 0x7f0e60009af0 2024-09-12T01:33:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.573+0000 7f0e1e7fc640 1 -- 172.21.15.16:0/4215659587 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e2406bf30 msgr2=0x7f0e2406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.573+0000 7f0e1e7fc640 1 --2- 172.21.15.16:0/4215659587 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e2406bf30 0x7f0e2406e3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0e28034000 tx=0x7f0e2802e000 comp rx=0 tx=0).stop 2024-09-12T01:33:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.573+0000 7f0e1e7fc640 1 -- 172.21.15.16:0/4215659587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e380af5d0 msgr2=0x7f0e38153ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.574+0000 7f0e1e7fc640 1 --2- 172.21.15.16:0/4215659587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e380af5d0 0x7f0e38153ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009ed0 tx=0x7f0e2c00ef00 comp rx=0 tx=0).stop 2024-09-12T01:33:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.574+0000 7f0e5e7fc640 1 -- 172.21.15.16:0/3705770502 <== mon.2 v2:172.21.15.33:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e64079db0 con 0x7f0e60009af0 2024-09-12T01:33:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.574+0000 7f0e1e7fc640 1 -- 172.21.15.16:0/4215659587 shutdown_connections 2024-09-12T01:33:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.574+0000 7f0e1e7fc640 1 --2- 172.21.15.16:0/4215659587 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e2406bf30 0x7f0e2406e3f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.574+0000 7f0e1e7fc640 1 --2- 172.21.15.16:0/4215659587 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e380cd360 0x7f0e380c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.574+0000 7f0e1e7fc640 1 --2- 172.21.15.16:0/4215659587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e380ccf80 0x7f0e38154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.574+0000 7f0e1e7fc640 1 --2- 172.21.15.16:0/4215659587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e380af5d0 0x7f0e38153ac0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.574+0000 7f0e1e7fc640 1 -- 172.21.15.16:0/4215659587 >> 172.21.15.16:0/4215659587 conn(0x7f0e380b1af0 msgr2=0x7f0e380b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.577+0000 7f68d57fa640 1 -- 172.21.15.16:0/3410298768 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f68e80008d0 con 0x7f68e80ea670 2024-09-12T01:33:34.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.574+0000 7f0e1e7fc640 1 -- 172.21.15.16:0/4215659587 shutdown_connections 2024-09-12T01:33:34.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.574+0000 7f0e1e7fc640 1 -- 172.21.15.16:0/4215659587 wait complete. 2024-09-12T01:33:34.577 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2024-09-12T01:33:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.577+0000 7f5c4b427640 1 -- 172.21.15.16:0/2642019771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5c3c004ba0 con 0x7f5c3c00b770 2024-09-12T01:33:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.577+0000 7f68d77fe640 1 -- 172.21.15.16:0/3410298768 <== mon.2 v2:172.21.15.33:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f68e4078cb0 con 0x7f68e80ea670 2024-09-12T01:33:34.578 INFO:teuthology.orchestra.run.smithi016.stdout:55834574855 2024-09-12T01:33:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.578+0000 7f849aa45640 1 -- 172.21.15.16:0/4032448592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8474005580 con 0x7f84940c5670 2024-09-12T01:33:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.578+0000 7f5c39ffb640 1 -- 172.21.15.16:0/2642019771 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5c40058fd0 con 0x7f5c3c00b770 2024-09-12T01:33:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.578+0000 7f8490ff9640 1 -- 172.21.15.16:0/4032448592 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8488079e90 con 0x7f84940c5670 2024-09-12T01:33:34.579 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-12T01:33:34.579 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-12T01:33:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.580+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/3705770502 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e4c06be30 msgr2=0x7f0e4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.580+0000 7f0e6ecaf640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e4c06be30 0x7f0e4c06e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e6004e4b0 tx=0x7f0e58007990 comp rx=0 tx=0).stop 2024-09-12T01:33:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.580+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/3705770502 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e60009af0 msgr2=0x7f0e6003a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.580+0000 7f0e6ecaf640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e60009af0 0x7f0e6003a060 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400c3f0 tx=0x7f0e640046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.580+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/3705770502 shutdown_connections 2024-09-12T01:33:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.580+0000 7f0e6ecaf640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e4c06be30 0x7f0e4c06e2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.580+0000 7f0e6ecaf640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e600dec70 0x7f0e6004db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.580+0000 7f0e6ecaf640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600323c0 0x7f0e600373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.580+0000 7f0e6ecaf640 1 --2- 172.21.15.16:0/3705770502 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e60009af0 0x7f0e6003a060 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.580+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/3705770502 >> 172.21.15.16:0/3705770502 conn(0x7f0e60036090 msgr2=0x7f0e600352a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.581+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/3705770502 shutdown_connections 2024-09-12T01:33:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.581+0000 7f0e6ecaf640 1 -- 172.21.15.16:0/3705770502 wait complete. 2024-09-12T01:33:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.581+0000 7f68d57fa640 1 -- 172.21.15.16:0/3410298768 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f68dc06c060 msgr2=0x7f68dc06e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.581+0000 7f68d57fa640 1 --2- 172.21.15.16:0/3410298768 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f68dc06c060 0x7f68dc06e520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f68e0006a30 tx=0x7f68e00069c0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.581+0000 7f68d57fa640 1 -- 172.21.15.16:0/3410298768 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68e80ea670 msgr2=0x7f68e8049070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.581+0000 7f68d57fa640 1 --2- 172.21.15.16:0/3410298768 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68e80ea670 0x7f68e8049070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f68e400a120 tx=0x7f68e4004730 comp rx=0 tx=0).stop 2024-09-12T01:33:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f68d57fa640 1 -- 172.21.15.16:0/3410298768 shutdown_connections 2024-09-12T01:33:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f68d57fa640 1 --2- 172.21.15.16:0/3410298768 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f68dc06c060 0x7f68dc06e520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f68d57fa640 1 --2- 172.21.15.16:0/3410298768 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f68e80ea670 0x7f68e8049070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f5c4b427640 1 -- 172.21.15.16:0/2642019771 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5c2806be60 msgr2=0x7f5c2806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f5c4b427640 1 --2- 172.21.15.16:0/2642019771 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5c2806be60 0x7f5c2806e320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c004f30 tx=0x7f5c34008040 comp rx=0 tx=0).stop 2024-09-12T01:33:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f5c4b427640 1 -- 172.21.15.16:0/2642019771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c3c00b770 msgr2=0x7f5c3c005330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f5c4b427640 1 --2- 172.21.15.16:0/2642019771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c3c00b770 0x7f5c3c005330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c480 tx=0x7f5c4000c940 comp rx=0 tx=0).stop 2024-09-12T01:33:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f68d57fa640 1 --2- 172.21.15.16:0/3410298768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e804bbd0 0x7f68e80ea130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f68d57fa640 1 --2- 172.21.15.16:0/3410298768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68d8005020 0x7f68e80213a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f68d57fa640 1 -- 172.21.15.16:0/3410298768 >> 172.21.15.16:0/3410298768 conn(0x7f68e801ea50 msgr2=0x7f68e80d9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f5c4b427640 1 -- 172.21.15.16:0/2642019771 shutdown_connections 2024-09-12T01:33:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f5c4b427640 1 --2- 172.21.15.16:0/2642019771 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5c2806be60 0x7f5c2806e320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f5c4b427640 1 --2- 172.21.15.16:0/2642019771 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c3c039790 0x7f5c3c0f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f5c4b427640 1 --2- 172.21.15.16:0/2642019771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c3c036020 0x7f5c3c0f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f5c4b427640 1 --2- 172.21.15.16:0/2642019771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c3c00b770 0x7f5c3c005330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f5c4b427640 1 -- 172.21.15.16:0/2642019771 >> 172.21.15.16:0/2642019771 conn(0x7f5c3c01d930 msgr2=0x7f5c3c0f9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f849aa45640 1 -- 172.21.15.16:0/4032448592 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f847806bf30 msgr2=0x7f847806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f8f7d01e640 1 -- 172.21.15.16:0/1146608636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8f58005580 con 0x7f8f780af5d0 2024-09-12T01:33:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.582+0000 7f5c4b427640 1 -- 172.21.15.16:0/2642019771 shutdown_connections 2024-09-12T01:33:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f849aa45640 1 --2- 172.21.15.16:0/4032448592 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f847806bf30 0x7f847806e3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8480005880 tx=0x7f8480007a20 comp rx=0 tx=0).stop 2024-09-12T01:33:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f5c4b427640 1 -- 172.21.15.16:0/2642019771 wait complete. 2024-09-12T01:33:34.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f849aa45640 1 -- 172.21.15.16:0/4032448592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84940c5670 msgr2=0x7f84940ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f849aa45640 1 --2- 172.21.15.16:0/4032448592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84940c5670 0x7f84940ca5c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84880098c0 tx=0x7f84880046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f849aa45640 1 -- 172.21.15.16:0/4032448592 shutdown_connections 2024-09-12T01:33:34.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f68d57fa640 1 -- 172.21.15.16:0/3410298768 shutdown_connections 2024-09-12T01:33:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f68d57fa640 1 -- 172.21.15.16:0/3410298768 wait complete. 2024-09-12T01:33:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f849aa45640 1 --2- 172.21.15.16:0/4032448592 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f847806bf30 0x7f847806e3f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f849aa45640 1 --2- 172.21.15.16:0/4032448592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f849414e840 0x7f849414d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f849aa45640 1 --2- 172.21.15.16:0/4032448592 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8494166cb0 0x7f8494151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f849aa45640 1 --2- 172.21.15.16:0/4032448592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84940c5670 0x7f84940ca5c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f849aa45640 1 -- 172.21.15.16:0/4032448592 >> 172.21.15.16:0/4032448592 conn(0x7f849409b2e0 msgr2=0x7f84940b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f8f5f7fe640 1 -- 172.21.15.16:0/1146608636 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8f6805d230 con 0x7f8f780af5d0 2024-09-12T01:33:34.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f849aa45640 1 -- 172.21.15.16:0/4032448592 shutdown_connections 2024-09-12T01:33:34.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.583+0000 7f849aa45640 1 -- 172.21.15.16:0/4032448592 wait complete. 2024-09-12T01:33:34.588 INFO:teuthology.orchestra.run.smithi016.stdout:60129542152 2024-09-12T01:33:34.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.587+0000 7f8f7d01e640 1 -- 172.21.15.16:0/1146608636 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8f6406be80 msgr2=0x7f8f6406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.587+0000 7f8f7d01e640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8f6406be80 0x7f8f6406e340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f780cf6f0 tx=0x7f8f6c007a40 comp rx=0 tx=0).stop 2024-09-12T01:33:34.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.587+0000 7f8f7d01e640 1 -- 172.21.15.16:0/1146608636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f780af5d0 msgr2=0x7f8f781532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.587+0000 7f8f7d01e640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f780af5d0 0x7f8f781532c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800e870 tx=0x7f8f6800ed30 comp rx=0 tx=0).stop 2024-09-12T01:33:34.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.587+0000 7f8f7d01e640 1 -- 172.21.15.16:0/1146608636 shutdown_connections 2024-09-12T01:33:34.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.587+0000 7f8f7d01e640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8f6406be80 0x7f8f6406e340 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f780cf6f0 tx=0x7f8f6c007a40 comp rx=0 tx=0).stop 2024-09-12T01:33:34.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.587+0000 7f8f7d01e640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8f780ccf80 0x7f8f781515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.6 2024-09-12T01:33:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.587+0000 7f8f7d01e640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f780b25e0 0x7f8f780c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.587+0000 7f8f7d01e640 1 --2- 172.21.15.16:0/1146608636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f780af5d0 0x7f8f781532c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.587+0000 7f8f7d01e640 1 -- 172.21.15.16:0/1146608636 >> 172.21.15.16:0/1146608636 conn(0x7f8f780b1af0 msgr2=0x7f8f780b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.588+0000 7f8f7d01e640 1 -- 172.21.15.16:0/1146608636 shutdown_connections 2024-09-12T01:33:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.588+0000 7f8f7d01e640 1 -- 172.21.15.16:0/1146608636 wait complete. 2024-09-12T01:33:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.589+0000 7fcb3988b640 1 -- 172.21.15.16:0/1754262119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcb2c0527e0 con 0x7fcb2c04df90 2024-09-12T01:33:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.590+0000 7fcb1bfff640 1 -- 172.21.15.16:0/1754262119 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcb2409c190 con 0x7fcb2c04df90 2024-09-12T01:33:34.592 INFO:teuthology.orchestra.run.smithi016.stdout:60129542153 2024-09-12T01:33:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.593+0000 7fcb3988b640 1 -- 172.21.15.16:0/1754262119 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fcb2006be30 msgr2=0x7fcb2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.593+0000 7fcb3988b640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fcb2006be30 0x7fcb2006e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0df0d0 tx=0x7fcb28009040 comp rx=0 tx=0).stop 2024-09-12T01:33:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.594+0000 7fcb3988b640 1 -- 172.21.15.16:0/1754262119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb2c04df90 msgr2=0x7fcb2c178440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.594+0000 7fcb3988b640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb2c04df90 0x7fcb2c178440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400fb30 tx=0x7fcb24009210 comp rx=0 tx=0).stop 2024-09-12T01:33:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.594+0000 7fcb3988b640 1 -- 172.21.15.16:0/1754262119 shutdown_connections 2024-09-12T01:33:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.594+0000 7fcb3988b640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fcb2006be30 0x7fcb2006e2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.594+0000 7fcb3988b640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fcb2c031950 0x7fcb2c031dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.594+0000 7fcb3988b640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb2c0c74b0 0x7fcb2c0c5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.594+0000 7fcb3988b640 1 --2- 172.21.15.16:0/1754262119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb2c04df90 0x7fcb2c178440 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.594+0000 7fcb3988b640 1 -- 172.21.15.16:0/1754262119 >> 172.21.15.16:0/1754262119 conn(0x7fcb2c01d930 msgr2=0x7fcb2c04fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.594+0000 7fcb3988b640 1 -- 172.21.15.16:0/1754262119 shutdown_connections 2024-09-12T01:33:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:34.594+0000 7fcb3988b640 1 -- 172.21.15.16:0/1754262119 wait complete. 2024-09-12T01:33:34.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.4 2024-09-12T01:33:34.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.2 2024-09-12T01:33:34.603 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.7 2024-09-12T01:33:34.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.5 2024-09-12T01:33:34.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.0 2024-09-12T01:33:34.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.1 2024-09-12T01:33:34.611 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.3 2024-09-12T01:33:35.591 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T01:33:35.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T01:33:35.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T01:33:35.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T01:33:35.608 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T01:33:35.608 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T01:33:35.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T01:33:35.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T01:33:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.768+0000 7f160693d640 1 Processor -- start 2024-09-12T01:33:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.769+0000 7f160693d640 1 -- start start 2024-09-12T01:33:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.771+0000 7f160693d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16000b26c0 0x7f1600072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.771+0000 7f160693d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f160015dea0 0x7f160015e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.771+0000 7f160693d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f160006deb0 0x7f160006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.771+0000 7f160693d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f160006f840 con 0x7f16000b26c0 2024-09-12T01:33:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.771+0000 7f160693d640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f160015e800 con 0x7f160006deb0 2024-09-12T01:33:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.771+0000 7f160693d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f1600161210 con 0x7f160015dea0 2024-09-12T01:33:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.771+0000 7f160593b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16000b26c0 0x7f1600072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.771+0000 7f160593b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16000b26c0 0x7f1600072710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42328/0 (socket says 172.21.15.16:42328) 2024-09-12T01:33:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.772+0000 7f160513a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f160015dea0 0x7f160015e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.772+0000 7f160513a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f160015dea0 0x7f160015e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44540/0 (socket says 172.21.15.16:44540) 2024-09-12T01:33:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.772+0000 7f160513a640 1 -- 172.21.15.16:0/661856310 learned_addr learned my addr 172.21.15.16:0/661856310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.772+0000 7f160613c640 1 --2- 172.21.15.16:0/661856310 >> v2:172.21.15.33:3300/0 conn(0x7f160006deb0 0x7f160006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.772+0000 7f160513a640 1 -- 172.21.15.16:0/661856310 >> v2:172.21.15.33:3300/0 conn(0x7f160006deb0 msgr2=0x7f160006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.772+0000 7f160513a640 1 --2- 172.21.15.16:0/661856310 >> v2:172.21.15.33:3300/0 conn(0x7f160006deb0 0x7f160006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.772+0000 7f160513a640 1 -- 172.21.15.16:0/661856310 >> v2:172.21.15.16:3300/0 conn(0x7f16000b26c0 msgr2=0x7f1600072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.772+0000 7f160513a640 1 --2- 172.21.15.16:0/661856310 >> v2:172.21.15.16:3300/0 conn(0x7f16000b26c0 0x7f1600072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.772+0000 7f160513a640 1 -- 172.21.15.16:0/661856310 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160006ef10 con 0x7f160015dea0 2024-09-12T01:33:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.773+0000 7f160513a640 1 --2- 172.21.15.16:0/661856310 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f160015dea0 0x7f160015e2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16000688c0 tx=0x7f15fc02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dad5b00a9addf53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.773+0000 7f160593b640 1 --2- 172.21.15.16:0/661856310 >> v2:172.21.15.16:3300/0 conn(0x7f16000b26c0 0x7f1600072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.774+0000 7f160613c640 1 --2- 172.21.15.16:0/661856310 >> v2:172.21.15.33:3300/0 conn(0x7f160006deb0 0x7f160006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.774+0000 7f1604939640 1 -- 172.21.15.16:0/661856310 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc037070 con 0x7f160015dea0 2024-09-12T01:33:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.774+0000 7f1604939640 1 -- 172.21.15.16:0/661856310 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15fc032440 con 0x7f160015dea0 2024-09-12T01:33:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.774+0000 7f1604939640 1 -- 172.21.15.16:0/661856310 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0327f0 con 0x7f160015dea0 2024-09-12T01:33:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.775+0000 7f160693d640 1 -- 172.21.15.16:0/661856310 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f160015dea0 msgr2=0x7f160015e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.775+0000 7f160693d640 1 --2- 172.21.15.16:0/661856310 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f160015dea0 0x7f160015e2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16000688c0 tx=0x7f15fc02a250 comp rx=0 tx=0).stop 2024-09-12T01:33:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.775+0000 7f160693d640 1 -- 172.21.15.16:0/661856310 shutdown_connections 2024-09-12T01:33:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.775+0000 7f160693d640 1 --2- 172.21.15.16:0/661856310 >> v2:172.21.15.33:3300/0 conn(0x7f160006deb0 0x7f160006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.775+0000 7f160693d640 1 --2- 172.21.15.16:0/661856310 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f160015dea0 0x7f160015e2c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.775+0000 7f160693d640 1 --2- 172.21.15.16:0/661856310 >> v2:172.21.15.16:3300/0 conn(0x7f16000b26c0 0x7f1600072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.775+0000 7f160693d640 1 -- 172.21.15.16:0/661856310 >> 172.21.15.16:0/661856310 conn(0x7f16000b1bd0 msgr2=0x7f16000b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.776+0000 7f160693d640 1 -- 172.21.15.16:0/661856310 shutdown_connections 2024-09-12T01:33:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.776+0000 7f160693d640 1 -- 172.21.15.16:0/661856310 wait complete. 2024-09-12T01:33:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.778+0000 7f160693d640 1 Processor -- start 2024-09-12T01:33:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.778+0000 7f160693d640 1 -- start start 2024-09-12T01:33:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.779+0000 7f160693d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160006deb0 0x7f160009d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.779+0000 7f160693d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000b26c0 0x7f160009aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.779+0000 7f160693d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f160015dea0 0x7f16001fbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.779+0000 7f160693d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1600072c50 con 0x7f16000b26c0 2024-09-12T01:33:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.779+0000 7f160693d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f160006f4d0 con 0x7f160006deb0 2024-09-12T01:33:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.779+0000 7f160693d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f160006e870 con 0x7f160015dea0 2024-09-12T01:33:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.780+0000 7f160513a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000b26c0 0x7f160009aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.780+0000 7f160513a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000b26c0 0x7f160009aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42336/0 (socket says 172.21.15.16:42336) 2024-09-12T01:33:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.780+0000 7f160513a640 1 -- 172.21.15.16:0/1211004463 learned_addr learned my addr 172.21.15.16:0/1211004463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.780+0000 7f160513a640 1 -- 172.21.15.16:0/1211004463 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f160015dea0 msgr2=0x7f16001fbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.780+0000 7f160513a640 1 --2- 172.21.15.16:0/1211004463 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f160015dea0 0x7f16001fbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.780+0000 7f160513a640 1 -- 172.21.15.16:0/1211004463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160006deb0 msgr2=0x7f160009d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.780+0000 7f160513a640 1 --2- 172.21.15.16:0/1211004463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160006deb0 0x7f160009d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.780+0000 7f160513a640 1 -- 172.21.15.16:0/1211004463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16000b0ae0 con 0x7f16000b26c0 2024-09-12T01:33:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.781+0000 7f160513a640 1 --2- 172.21.15.16:0/1211004463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000b26c0 0x7f160009aca0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0097f0 tx=0x7f15fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.781+0000 7f15f67fc640 1 -- 172.21.15.16:0/1211004463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc037070 con 0x7f16000b26c0 2024-09-12T01:33:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.782+0000 7f160693d640 1 -- 172.21.15.16:0/1211004463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16000b0dc0 con 0x7f16000b26c0 2024-09-12T01:33:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.782+0000 7f160693d640 1 -- 172.21.15.16:0/1211004463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160009d9c0 con 0x7f16000b26c0 2024-09-12T01:33:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.785+0000 7f160693d640 1 -- 172.21.15.16:0/1211004463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16000734d0 con 0x7f16000b26c0 2024-09-12T01:33:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.795+0000 7f15f67fc640 1 -- 172.21.15.16:0/1211004463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15fc041070 con 0x7f16000b26c0 2024-09-12T01:33:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.795+0000 7f15f67fc640 1 -- 172.21.15.16:0/1211004463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc032830 con 0x7f16000b26c0 2024-09-12T01:33:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.796+0000 7f15f67fc640 1 -- 172.21.15.16:0/1211004463 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15fc0329d0 con 0x7f16000b26c0 2024-09-12T01:33:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.796+0000 7f15f67fc640 1 --2- 172.21.15.16:0/1211004463 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f15e406bf30 0x7f15e406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.797+0000 7f160593b640 1 --2- 172.21.15.16:0/1211004463 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f15e406bf30 0x7f15e406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.797+0000 7f15f67fc640 1 -- 172.21.15.16:0/1211004463 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f15fc0ab580 con 0x7f16000b26c0 2024-09-12T01:33:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.797+0000 7f15f67fc640 1 -- 172.21.15.16:0/1211004463 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f16000734d0 con 0x7f16000b26c0 2024-09-12T01:33:35.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.799+0000 7f160593b640 1 --2- 172.21.15.16:0/1211004463 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f15e406bf30 0x7f15e406e3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f15f0006fd0 tx=0x7f15f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.815+0000 7f6be6201640 1 Processor -- start 2024-09-12T01:33:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.816+0000 7f6be6201640 1 -- start start 2024-09-12T01:33:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.818+0000 7f6be6201640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6be00cdde0 0x7f6be0163670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.818+0000 7f6be6201640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be0163bb0 con 0x7f6be00ce1c0 2024-09-12T01:33:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.818+0000 7f6be6201640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be006dc10 con 0x7f6be006f750 2024-09-12T01:33:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.818+0000 7f6be6201640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f6be006e820 con 0x7f6be00cdde0 2024-09-12T01:33:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.818+0000 7f217e03c640 1 Processor -- start 2024-09-12T01:33:35.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.818+0000 7f6bdffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6be00ce1c0 0x7f6be00ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53712/0 (socket says 172.21.15.16:53712) 2024-09-12T01:33:35.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.818+0000 7f6bdffff640 1 -- 172.21.15.16:0/3277811028 learned_addr learned my addr 172.21.15.16:0/3277811028 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:35.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.819+0000 7f6bdf7fe640 1 --2- 172.21.15.16:0/3277811028 >> v2:172.21.15.33:3300/0 conn(0x7f6be00cdde0 0x7f6be0163670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.819+0000 7f6bdf7fe640 1 -- 172.21.15.16:0/3277811028 >> v1:172.21.15.33:6789/0 conn(0x7f6be006f750 legacy=0x7f6be00cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.819+0000 7f6bdf7fe640 1 -- 172.21.15.16:0/3277811028 >> v1:172.21.15.16:6789/0 conn(0x7f6be00ce1c0 legacy=0x7f6be00ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.819+0000 7f6bdf7fe640 1 -- 172.21.15.16:0/3277811028 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be006def0 con 0x7f6be00cdde0 2024-09-12T01:33:35.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.820+0000 7fc3cdf5f640 1 Processor -- start 2024-09-12T01:33:35.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.818+0000 7f217e03c640 1 -- start start 2024-09-12T01:33:35.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.820+0000 7f217e03c640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21780ccf80 0x7f21780af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.820+0000 7f217e03c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21780af5d0 0x7f2178159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.820+0000 7f217e03c640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f217815e260 con 0x7f21780cd360 2024-09-12T01:33:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.820+0000 7f217e03c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f217815ede0 con 0x7f21780af5d0 2024-09-12T01:33:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.820+0000 7f217e03c640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f217815e540 con 0x7f21780ccf80 2024-09-12T01:33:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.820+0000 7fc3cdf5f640 1 -- start start 2024-09-12T01:33:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.820+0000 7f6bdf7fe640 1 --2- 172.21.15.16:0/3277811028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be00cdde0 0x7f6be0163670 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009d90 tx=0x7f6bd802a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a0992f76cfa84ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.820+0000 7f2176ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21780af5d0 0x7f2178159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.820+0000 7f2176ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21780af5d0 0x7f2178159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42344/0 (socket says 172.21.15.16:42344) 2024-09-12T01:33:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.820+0000 7f2176ffd640 1 -- 172.21.15.16:0/2251684151 learned_addr learned my addr 172.21.15.16:0/2251684151 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.820+0000 7f21777fe640 1 --2- 172.21.15.16:0/2251684151 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21780ccf80 0x7f21780af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7f21767fc640 1 -- 172.21.15.16:0/2251684151 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1636594251 0 0) 0x7f217815e260 con 0x7f21780cd360 2024-09-12T01:33:35.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7f21767fc640 1 -- 172.21.15.16:0/2251684151 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2160003520 con 0x7f21780cd360 2024-09-12T01:33:35.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7f6bde7fc640 1 -- 172.21.15.16:0/3277811028 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8007740 con 0x7f6be00cdde0 2024-09-12T01:33:35.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7f6bde7fc640 1 -- 172.21.15.16:0/3277811028 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd80078e0 con 0x7f6be00cdde0 2024-09-12T01:33:35.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7f6bde7fc640 1 -- 172.21.15.16:0/3277811028 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8007cf0 con 0x7f6be00cdde0 2024-09-12T01:33:35.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7f21777fe640 1 -- 172.21.15.16:0/2251684151 >> v2:172.21.15.16:3300/0 conn(0x7f21780af5d0 msgr2=0x7f2178159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:35.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7f21767fc640 1 -- 172.21.15.16:0/2251684151 <== mon.1 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2660412590 0 0) 0x7f2160003520 con 0x7f21780cd360 2024-09-12T01:33:35.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7f21777fe640 1 --2- 172.21.15.16:0/2251684151 >> v2:172.21.15.16:3300/0 conn(0x7f21780af5d0 0x7f2178159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7f21777fe640 1 -- 172.21.15.16:0/2251684151 >> v1:172.21.15.33:6789/0 conn(0x7f21780cd360 legacy=0x7f217806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7f21777fe640 1 -- 172.21.15.16:0/2251684151 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217815e820 con 0x7f21780ccf80 2024-09-12T01:33:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.822+0000 7f6be6201640 1 -- 172.21.15.16:0/3277811028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be00cdde0 msgr2=0x7f6be0163670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.822+0000 7f6be6201640 1 --2- 172.21.15.16:0/3277811028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be00cdde0 0x7f6be0163670 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009d90 tx=0x7f6bd802a6a0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7fc3cdf5f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc3c815a270 0x7fc3c80cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7fc3cdf5f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c80cd630 con 0x7fc3c806bfc0 2024-09-12T01:33:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7fc3cdf5f640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c806f180 con 0x7fc3c806eca0 2024-09-12T01:33:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.821+0000 7fc3cdf5f640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c806f460 con 0x7fc3c815a270 2024-09-12T01:33:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.822+0000 7fc3c6ffd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc3c806eca0 0x7fc3c80cc690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:33564/0 (socket says 172.21.15.16:33564) 2024-09-12T01:33:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.822+0000 7fc3c6ffd640 1 -- 172.21.15.16:0/2722104725 learned_addr learned my addr 172.21.15.16:0/2722104725 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.822+0000 7f6be6201640 1 -- 172.21.15.16:0/3277811028 shutdown_connections 2024-09-12T01:33:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.822+0000 7f6be6201640 1 --2- 172.21.15.16:0/3277811028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be00cdde0 0x7f6be0163670 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.822+0000 7f6be6201640 1 -- 172.21.15.16:0/3277811028 >> 172.21.15.16:0/3277811028 conn(0x7f6be00b1bd0 msgr2=0x7f6be00b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.822+0000 7fc3c77fe640 1 --2- 172.21.15.16:0/2722104725 >> v2:172.21.15.33:3300/0 conn(0x7fc3c815a270 0x7fc3c80cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.822+0000 7f6be6201640 1 -- 172.21.15.16:0/3277811028 shutdown_connections 2024-09-12T01:33:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.822+0000 7f6be6201640 1 -- 172.21.15.16:0/3277811028 wait complete. 2024-09-12T01:33:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.822+0000 7f714f3cc640 1 Processor -- start 2024-09-12T01:33:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.823+0000 7fc3c77fe640 1 -- 172.21.15.16:0/2722104725 >> v1:172.21.15.33:6789/0 conn(0x7fc3c806eca0 legacy=0x7fc3c80cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.823+0000 7fc3c77fe640 1 -- 172.21.15.16:0/2722104725 >> v1:172.21.15.16:6789/0 conn(0x7fc3c806bfc0 legacy=0x7fc3c80b4ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:33:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.822+0000 7fc3c67fc640 1 -- 172.21.15.16:0/2722104725 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 916426154 0 0) 0x7fc3c806f180 con 0x7fc3c806eca0 2024-09-12T01:33:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.823+0000 7fc3c77fe640 1 -- 172.21.15.16:0/2722104725 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c806f740 con 0x7fc3c815a270 2024-09-12T01:33:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.823+0000 7f8e7753f640 1 Processor -- start 2024-09-12T01:33:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.823+0000 7f8e7753f640 1 -- start start 2024-09-12T01:33:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.823+0000 7f21777fe640 1 --2- 172.21.15.16:0/2251684151 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21780ccf80 0x7f21780af090 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f216800dbc0 tx=0x7f2168032820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dff0a36edf3dbca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.823+0000 7f21767fc640 1 -- 172.21.15.16:0/2251684151 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216803e070 con 0x7f21780ccf80 2024-09-12T01:33:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.823+0000 7f21767fc640 1 -- 172.21.15.16:0/2251684151 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f216800b450 con 0x7f21780ccf80 2024-09-12T01:33:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.824+0000 7f21767fc640 1 -- 172.21.15.16:0/2251684151 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c001670 con 0x7f21780ccf80 2024-09-12T01:33:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.824+0000 7f6be6201640 1 Processor -- start 2024-09-12T01:33:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.823+0000 7f9bfe394640 1 Processor -- start 2024-09-12T01:33:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.824+0000 7ff66e178640 1 Processor -- start 2024-09-12T01:33:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.824+0000 7f6be6201640 1 -- start start 2024-09-12T01:33:35.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.824+0000 7ff66e178640 1 -- start start 2024-09-12T01:33:35.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.823+0000 7f9bfe394640 1 -- start start 2024-09-12T01:33:35.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.824+0000 7fc3c77fe640 1 --2- 172.21.15.16:0/2722104725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c815a270 0x7fc3c80cd0f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc0097d0 tx=0x7fc3bc02b210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5e9671ea53fc5d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.823+0000 7f714f3cc640 1 -- start start 2024-09-12T01:33:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f217e03c640 1 -- 172.21.15.16:0/2251684151 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21780ccf80 msgr2=0x7f21780af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f217e03c640 1 --2- 172.21.15.16:0/2251684151 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21780ccf80 0x7f21780af090 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f216800dbc0 tx=0x7f2168032820 comp rx=0 tx=0).stop 2024-09-12T01:33:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.824+0000 7f714f3cc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f714806de00 0x7f7148072e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f8e7753f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e70145a80 0x7f8e700ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f8e7753f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8e700af4d0 0x7f8e7006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f8e7753f640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e70072d10 con 0x7f8e70072520 2024-09-12T01:33:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f217e03c640 1 -- 172.21.15.16:0/2251684151 shutdown_connections 2024-09-12T01:33:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f217e03c640 1 --2- 172.21.15.16:0/2251684151 >> v2:172.21.15.16:3300/0 conn(0x7f21780af5d0 0x7f2178159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f217e03c640 1 --2- 172.21.15.16:0/2251684151 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21780ccf80 0x7f21780af090 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f217e03c640 1 -- 172.21.15.16:0/2251684151 >> 172.21.15.16:0/2251684151 conn(0x7f21780b1af0 msgr2=0x7f21780b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f8e7753f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8e700ad9c0 con 0x7f8e70145a80 2024-09-12T01:33:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f8e7753f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f8e700735b0 con 0x7f8e700af4d0 2024-09-12T01:33:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.824+0000 7f714f3cc640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7148071fb0 0x7f7148162280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f217e03c640 1 -- 172.21.15.16:0/2251684151 shutdown_connections 2024-09-12T01:33:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f217e03c640 1 -- 172.21.15.16:0/2251684151 wait complete. 2024-09-12T01:33:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7fc3c67fc640 1 -- 172.21.15.16:0/2722104725 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc038070 con 0x7fc3c815a270 2024-09-12T01:33:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7fc3c67fc640 1 -- 172.21.15.16:0/2722104725 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3bc033440 con 0x7fc3c815a270 2024-09-12T01:33:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7fc3c67fc640 1 -- 172.21.15.16:0/2722104725 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc033810 con 0x7fc3c815a270 2024-09-12T01:33:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f714dbc9640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7148071fb0 0x7f7148162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f714dbc9640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7148071fb0 0x7f7148162280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45748/0 (socket says 172.21.15.16:45748) 2024-09-12T01:33:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f714dbc9640 1 -- 172.21.15.16:0/1791883448 learned_addr learned my addr 172.21.15.16:0/1791883448 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f8e7653d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e70145a80 0x7f8e700ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f8e7653d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e70145a80 0x7f8e700ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42350/0 (socket says 172.21.15.16:42350) 2024-09-12T01:33:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f8e7653d640 1 -- 172.21.15.16:0/1587163008 learned_addr learned my addr 172.21.15.16:0/1587163008 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f8e76d3e640 1 --2- 172.21.15.16:0/1587163008 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8e700af4d0 0x7f8e7006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f714f3cc640 1 -- 172.21.15.16:0/1791883448 --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7148073760 con 0x7f714806e1e0 2024-09-12T01:33:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f714f3cc640 1 -- 172.21.15.16:0/1791883448 --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7148073a40 con 0x7f714806de00 2024-09-12T01:33:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f714f3cc640 1 -- 172.21.15.16:0/1791883448 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f71480b36c0 con 0x7f7148071fb0 2024-09-12T01:33:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.825+0000 7f6be6201640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6be006f750 0x7f6be009c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6be6201640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be00cdde0 0x7f6be00cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6be6201640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be00ce1c0 0x7f6be00cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6be6201640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6be016d580 con 0x7f6be00ce1c0 2024-09-12T01:33:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6be6201640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6be006e4b0 con 0x7f6be00cdde0 2024-09-12T01:33:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6be6201640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f6be016d700 con 0x7f6be006f750 2024-09-12T01:33:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7ff66e178640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff6680ccf80 0x7ff6680af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6bdf7fe640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6be006f750 0x7f6be009c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6bdeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be00cdde0 0x7f6be00cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6bdf7fe640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6be006f750 0x7f6be009c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44568/0 (socket says 172.21.15.16:44568) 2024-09-12T01:33:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6bdf7fe640 1 -- 172.21.15.16:0/1618119498 learned_addr learned my addr 172.21.15.16:0/1618119498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6bdeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be00cdde0 0x7f6be00cbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45756/0 (socket says 172.21.15.16:45756) 2024-09-12T01:33:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7ff66e178640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff66815e260 con 0x7ff66815a530 2024-09-12T01:33:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6bdffff640 1 --2- 172.21.15.16:0/1618119498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be00ce1c0 0x7f6be00cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7ff66e178640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff66815e540 con 0x7ff6680af5d0 2024-09-12T01:33:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7ff66e178640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7ff66815ede0 con 0x7ff6680ccf80 2024-09-12T01:33:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6bdeffd640 1 -- 172.21.15.16:0/1618119498 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6be006f750 msgr2=0x7f6be009c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7ff6677fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff6680ccf80 0x7ff6680af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7ff666ffd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff6680af5d0 0x7ff668159e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:33582/0 (socket says 172.21.15.16:33582) 2024-09-12T01:33:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7ff666ffd640 1 -- 172.21.15.16:0/3635048473 learned_addr learned my addr 172.21.15.16:0/3635048473 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f8e7653d640 1 -- 172.21.15.16:0/1587163008 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8e700af4d0 msgr2=0x7f8e7006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f8e7653d640 1 --2- 172.21.15.16:0/1587163008 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8e700af4d0 0x7f8e7006e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f8e7653d640 1 -- 172.21.15.16:0/1587163008 >> v1:172.21.15.33:6789/0 conn(0x7f8e70072520 legacy=0x7f8e700aedc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T01:33:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f8e7653d640 1 -- 172.21.15.16:0/1587163008 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e70072ff0 con 0x7f8e70145a80 2024-09-12T01:33:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6bdeffd640 1 --2- 172.21.15.16:0/1618119498 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6be006f750 0x7f6be009c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6bdeffd640 1 -- 172.21.15.16:0/1618119498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be00ce1c0 msgr2=0x7f6be00cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f9bfe394640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9bf80d0b10 0x7f9bf8072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f9bfe394640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9bf806ee10 0x7f9bf806f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f9bfe394640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf806f760 con 0x7f9bf8144c20 2024-09-12T01:33:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f9bfe394640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf815efd0 con 0x7f9bf806ee10 2024-09-12T01:33:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f9bfe394640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf815f2b0 con 0x7f9bf80d0b10 2024-09-12T01:33:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f9bfd392640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9bf80d0b10 0x7f9bf8072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f9bfd392640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9bf80d0b10 0x7f9bf8072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45780/0 (socket says 172.21.15.16:45780) 2024-09-12T01:33:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f9bfd392640 1 -- 172.21.15.16:0/3912045347 learned_addr learned my addr 172.21.15.16:0/3912045347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f9beffff640 1 -- 172.21.15.16:0/3912045347 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542626795 0 0) 0x7f9be8001110 con 0x7f9bf8144c20 2024-09-12T01:33:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f9beffff640 1 -- 172.21.15.16:0/3912045347 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9be4003520 con 0x7f9bf8144c20 2024-09-12T01:33:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6bdeffd640 1 --2- 172.21.15.16:0/1618119498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be00ce1c0 0x7f6be00cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f6bdeffd640 1 -- 172.21.15.16:0/1618119498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be009b770 con 0x7f6be00cdde0 2024-09-12T01:33:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f9bfd392640 1 -- 172.21.15.16:0/3912045347 >> v2:172.21.15.16:3300/0 conn(0x7f9bf806ee10 msgr2=0x7f9bf806f220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:33:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f6bdeffd640 1 --2- 172.21.15.16:0/1618119498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be00cdde0 0x7f6be00cbba0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000c390 tx=0x7f6bd000c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f714e3ca640 1 --2- 172.21.15.16:0/1791883448 >> v2:172.21.15.16:3300/0 conn(0x7f714806de00 0x7f7148072e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f217e03c640 1 Processor -- start 2024-09-12T01:33:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f9beffff640 1 -- 172.21.15.16:0/3912045347 <== mon.1 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3606022867 0 0) 0x7f9be4003520 con 0x7f9bf8144c20 2024-09-12T01:33:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f714dbc9640 1 -- 172.21.15.16:0/1791883448 >> v2:172.21.15.16:3300/0 conn(0x7f714806de00 msgr2=0x7f7148072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:35.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f714dbc9640 1 --2- 172.21.15.16:0/1791883448 >> v2:172.21.15.16:3300/0 conn(0x7f714806de00 0x7f7148072e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f6bc7fff640 1 -- 172.21.15.16:0/1618119498 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0005220 con 0x7f6be00cdde0 2024-09-12T01:33:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f6bc7fff640 1 -- 172.21.15.16:0/1618119498 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd00053c0 con 0x7f6be00cdde0 2024-09-12T01:33:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f6be6201640 1 -- 172.21.15.16:0/1618119498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be009ba50 con 0x7f6be00cdde0 2024-09-12T01:33:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f714dbc9640 1 -- 172.21.15.16:0/1791883448 >> v1:172.21.15.16:6789/0 conn(0x7f714806e1e0 legacy=0x7f7148166340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f8e7653d640 1 --2- 172.21.15.16:0/1587163008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e70145a80 0x7f8e700ad480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009e50 tx=0x7f8e6802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3b0a0d95387cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f217e03c640 1 -- start start 2024-09-12T01:33:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f714dbc9640 1 -- 172.21.15.16:0/1791883448 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71480b3070 con 0x7f7148071fb0 2024-09-12T01:33:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.826+0000 7f714e3ca640 1 --2- 172.21.15.16:0/1791883448 >> v2:172.21.15.16:3300/0 conn(0x7f714806de00 0x7f7148072e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f714dbc9640 1 --2- 172.21.15.16:0/1791883448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7148071fb0 0x7f7148162280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7148069130 tx=0x7f713802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0a8eac720fb08a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2333944415 0 0) 0x7ff66815e540 con 0x7ff6680af5d0 2024-09-12T01:33:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff650003520 con 0x7ff6680af5d0 2024-09-12T01:33:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.828+0000 7f714d3c8640 1 -- 172.21.15.16:0/1791883448 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138037070 con 0x7f7148071fb0 2024-09-12T01:33:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.828+0000 7f714d3c8640 1 -- 172.21.15.16:0/1791883448 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7138032440 con 0x7f7148071fb0 2024-09-12T01:33:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.828+0000 7f714d3c8640 1 -- 172.21.15.16:0/1791883448 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138032810 con 0x7f7148071fb0 2024-09-12T01:33:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.828+0000 7f217e03c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21780af5d0 0x7f21780c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.828+0000 7f217e03c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21780ccf80 0x7f21780b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f217e03c640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21780cd360 0x7f21781f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f217e03c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21780b25e0 con 0x7f21780af5d0 2024-09-12T01:33:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f217e03c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f217815def0 con 0x7f21780ccf80 2024-09-12T01:33:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f6bc7fff640 1 -- 172.21.15.16:0/1618119498 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0007250 con 0x7f6be00cdde0 2024-09-12T01:33:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f6be6201640 1 -- 172.21.15.16:0/1618119498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be009c010 con 0x7f6be00cdde0 2024-09-12T01:33:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.828+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/2722104725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c815a270 msgr2=0x7fc3c80cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.828+0000 7fc3cdf5f640 1 --2- 172.21.15.16:0/2722104725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c815a270 0x7fc3c80cd0f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc0097d0 tx=0x7fc3bc02b210 comp rx=0 tx=0).stop 2024-09-12T01:33:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1262553023 0 0) 0x7ff66815e260 con 0x7ff66815a530 2024-09-12T01:33:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.828+0000 7f8e7553b640 1 -- 172.21.15.16:0/1587163008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68036070 con 0x7f8e70145a80 2024-09-12T01:33:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.828+0000 7f8e7553b640 1 -- 172.21.15.16:0/1587163008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e68031600 con 0x7f8e70145a80 2024-09-12T01:33:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.828+0000 7f8e7553b640 1 -- 172.21.15.16:0/1587163008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e680319d0 con 0x7f8e70145a80 2024-09-12T01:33:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f217e03c640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f21780b2760 con 0x7f21780cd360 2024-09-12T01:33:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff66815e540 con 0x7ff66815a530 2024-09-12T01:33:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 <== mon.1 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3609612712 0 0) 0x7ff650003520 con 0x7ff6680af5d0 2024-09-12T01:33:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 >> v2:172.21.15.33:3300/0 conn(0x7ff6680ccf80 msgr2=0x7ff6680af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 --2- 172.21.15.16:0/3635048473 >> v2:172.21.15.33:3300/0 conn(0x7ff6680ccf80 0x7ff6680af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 >> v1:172.21.15.16:6789/0 conn(0x7ff66815a530 legacy=0x7ff6680ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.827+0000 7f9bfdb93640 1 --2- 172.21.15.16:0/3912045347 >> v2:172.21.15.16:3300/0 conn(0x7f9bf806ee10 0x7f9bf806f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.828+0000 7f9bfd392640 1 --2- 172.21.15.16:0/3912045347 >> v2:172.21.15.16:3300/0 conn(0x7f9bf806ee10 0x7f9bf806f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.828+0000 7f9bfd392640 1 -- 172.21.15.16:0/3912045347 >> v1:172.21.15.33:6789/0 conn(0x7f9bf8144c20 legacy=0x7f9bf80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.828+0000 7f9bfd392640 1 -- 172.21.15.16:0/3912045347 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf815f590 con 0x7f9bf80d0b10 2024-09-12T01:33:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66815e820 con 0x7ff6680af5d0 2024-09-12T01:33:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f9bfdb93640 1 --2- 172.21.15.16:0/3912045347 >> v2:172.21.15.16:3300/0 conn(0x7f9bf806ee10 0x7f9bf806f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 <== mon.1 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7ff65c0015f0 con 0x7ff6680af5d0 2024-09-12T01:33:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f714f3cc640 1 -- 172.21.15.16:0/1791883448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7148071fb0 msgr2=0x7f7148162280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f714f3cc640 1 --2- 172.21.15.16:0/1791883448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7148071fb0 0x7f7148162280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7148069130 tx=0x7f713802a230 comp rx=0 tx=0).stop 2024-09-12T01:33:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f6be6201640 1 -- 172.21.15.16:0/1618119498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be0163670 con 0x7f6be00cdde0 2024-09-12T01:33:35.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 >> v1:172.21.15.33:6789/0 conn(0x7ff6680af5d0 legacy=0x7ff668159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 --2- 172.21.15.16:0/3635048473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650003bb0 0x7ff650003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f9bfd392640 1 --2- 172.21.15.16:0/3912045347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf80d0b10 0x7f9bf8072f70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8072d30 tx=0x7f9bf4033150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a25af3b1790d8a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 --2- 172.21.15.16:0/3635048473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff650004800 0x7ff650004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 --2- 172.21.15.16:0/3635048473 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff6500052e0 0x7ff65000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/2722104725 shutdown_connections 2024-09-12T01:33:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7fc3cdf5f640 1 --2- 172.21.15.16:0/2722104725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c815a270 0x7fc3c80cd0f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/2722104725 >> 172.21.15.16:0/2722104725 conn(0x7fc3c80b2510 msgr2=0x7fc3c80b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff66815ede0 con 0x7ff650003bb0 2024-09-12T01:33:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff65000c190 con 0x7ff650004800 2024-09-12T01:33:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f8e7753f640 1 -- 172.21.15.16:0/1587163008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e70145a80 msgr2=0x7f8e700ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f8e7753f640 1 --2- 172.21.15.16:0/1587163008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e70145a80 0x7f8e700ad480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009e50 tx=0x7f8e6802a640 comp rx=0 tx=0).stop 2024-09-12T01:33:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff65000c390 con 0x7ff6500052e0 2024-09-12T01:33:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f714f3cc640 1 -- 172.21.15.16:0/1791883448 shutdown_connections 2024-09-12T01:33:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f714f3cc640 1 --2- 172.21.15.16:0/1791883448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7148071fb0 0x7f7148162280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f714f3cc640 1 --2- 172.21.15.16:0/1791883448 >> v2:172.21.15.16:3300/0 conn(0x7f714806de00 0x7f7148072e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f2176ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21780ccf80 0x7f21780b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f2176ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21780ccf80 0x7f21780b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45788/0 (socket says 172.21.15.16:45788) 2024-09-12T01:33:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f714f3cc640 1 -- 172.21.15.16:0/1791883448 >> 172.21.15.16:0/1791883448 conn(0x7f71480b1bd0 msgr2=0x7f71480b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f8e7753f640 1 -- 172.21.15.16:0/1587163008 shutdown_connections 2024-09-12T01:33:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f8e7753f640 1 --2- 172.21.15.16:0/1587163008 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8e700af4d0 0x7f8e7006e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f8e7753f640 1 --2- 172.21.15.16:0/1587163008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e70145a80 0x7f8e700ad480 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f2176ffd640 1 -- 172.21.15.16:0/281296491 learned_addr learned my addr 172.21.15.16:0/281296491 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f2177fff640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21780cd360 0x7f21781f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f21777fe640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21780af5d0 0x7f21780c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff667fff640 1 --2- 172.21.15.16:0/3635048473 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff6500052e0 0x7ff65000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7f8e7753f640 1 -- 172.21.15.16:0/1587163008 >> 172.21.15.16:0/1587163008 conn(0x7f8e700b1bd0 msgr2=0x7f8e700b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7ff6677fe640 1 --2- 172.21.15.16:0/3635048473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650003bb0 0x7ff650003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7f8e7753f640 1 -- 172.21.15.16:0/1587163008 shutdown_connections 2024-09-12T01:33:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7f8e7753f640 1 -- 172.21.15.16:0/1587163008 wait complete. 2024-09-12T01:33:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.829+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/2722104725 shutdown_connections 2024-09-12T01:33:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/2722104725 wait complete. 2024-09-12T01:33:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7ff666ffd640 1 --2- 172.21.15.16:0/3635048473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff650004800 0x7ff650004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7f714f3cc640 1 -- 172.21.15.16:0/1791883448 shutdown_connections 2024-09-12T01:33:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7f2176ffd640 1 -- 172.21.15.16:0/281296491 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21780cd360 msgr2=0x7f21781f0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7f2176ffd640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21780cd360 0x7f21781f0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7f2176ffd640 1 -- 172.21.15.16:0/281296491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21780af5d0 msgr2=0x7f21780c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7f2176ffd640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21780af5d0 0x7f21780c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7f2176ffd640 1 -- 172.21.15.16:0/281296491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217809be90 con 0x7f21780ccf80 2024-09-12T01:33:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7f2177fff640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21780cd360 0x7f21781f0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7f2176ffd640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21780ccf80 0x7f21780b3b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f21700070b0 tx=0x7f217000e4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7f714f3cc640 1 -- 172.21.15.16:0/1791883448 wait complete. 2024-09-12T01:33:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7ff6677fe640 1 -- 172.21.15.16:0/3635048473 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff6500052e0 msgr2=0x7ff65000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7ff6677fe640 1 --2- 172.21.15.16:0/3635048473 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff6500052e0 0x7ff65000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7ff6677fe640 1 -- 172.21.15.16:0/3635048473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff650004800 msgr2=0x7ff650004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7ff6677fe640 1 --2- 172.21.15.16:0/3635048473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff650004800 0x7ff650004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.830+0000 7ff6677fe640 1 -- 172.21.15.16:0/3635048473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66815e820 con 0x7ff650003bb0 2024-09-12T01:33:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.831+0000 7ff667fff640 1 -- 172.21.15.16:0/3635048473 reap_dead start 2024-09-12T01:33:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.831+0000 7ff666ffd640 1 --2- 172.21.15.16:0/3635048473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff650004800 0x7ff650004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.831+0000 7f6bc7fff640 1 -- 172.21.15.16:0/1618119498 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6bd0019020 con 0x7f6be00cdde0 2024-09-12T01:33:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.831+0000 7f8e7753f640 1 Processor -- start 2024-09-12T01:33:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.831+0000 7f8e7753f640 1 -- start start 2024-09-12T01:33:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7f6bc7fff640 1 --2- 172.21.15.16:0/1618119498 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6bcc06bf00 0x7f6bcc06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7fc3cdf5f640 1 Processor -- start 2024-09-12T01:33:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7ff6677fe640 1 --2- 172.21.15.16:0/3635048473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650003bb0 0x7ff650003f90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff66000c5b0 tx=0x7ff650005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f230f91911aa835 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7f714f3cc640 1 Processor -- start 2024-09-12T01:33:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7f714f3cc640 1 -- start start 2024-09-12T01:33:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7f6bc7fff640 1 -- 172.21.15.16:0/1618119498 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6bd008a4e0 con 0x7f6be00cdde0 2024-09-12T01:33:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66003c070 con 0x7ff650003bb0 2024-09-12T01:33:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff660037410 con 0x7ff650003bb0 2024-09-12T01:33:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7ff6667fc640 1 -- 172.21.15.16:0/3635048473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66000c5f0 con 0x7ff650003bb0 2024-09-12T01:33:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7f6bdf7fe640 1 --2- 172.21.15.16:0/1618119498 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6bcc06bf00 0x7f6bcc06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7fc3cdf5f640 1 -- start start 2024-09-12T01:33:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7f8e7753f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8e70072520 0x7f8e7013e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7f8e7753f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e700af4d0 0x7f8e7013e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7f8e7753f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e70145a80 0x7f8e7009af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.832+0000 7f8e7753f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8e70072960 con 0x7f8e70145a80 2024-09-12T01:33:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f8e7753f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7006dd30 con 0x7f8e700af4d0 2024-09-12T01:33:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f8e7753f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f8e701402e0 con 0x7f8e70072520 2024-09-12T01:33:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f8e7653d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8e70072520 0x7f8e7013e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f8e7653d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8e70072520 0x7f8e7013e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:44600/0 (socket says 172.21.15.16:44600) 2024-09-12T01:33:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f8e7653d640 1 -- 172.21.15.16:0/3226049090 learned_addr learned my addr 172.21.15.16:0/3226049090 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f8e76d3e640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e70145a80 0x7f8e7009af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f714f3cc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f714806de00 0x7f714815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f8e75d3c640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e700af4d0 0x7f8e7013e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f714f3cc640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f714806e1e0 0x7f7148154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f6bdf7fe640 1 --2- 172.21.15.16:0/1618119498 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6bcc06bf00 0x7f6bcc06e3c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6bd802a6e0 tx=0x7f6bd802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f714f3cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7148071fb0 0x7f71481554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f714f3cc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7148073520 con 0x7f7148071fb0 2024-09-12T01:33:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f714f3cc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f71480733a0 con 0x7f714806de00 2024-09-12T01:33:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f714f3cc640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f71480aceb0 con 0x7f714806e1e0 2024-09-12T01:33:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f714e3ca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f714806de00 0x7f714815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f714e3ca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f714806de00 0x7f714815c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45828/0 (socket says 172.21.15.16:45828) 2024-09-12T01:33:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f714e3ca640 1 -- 172.21.15.16:0/1038335322 learned_addr learned my addr 172.21.15.16:0/1038335322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f8e7653d640 1 -- 172.21.15.16:0/3226049090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e700af4d0 msgr2=0x7f8e7013e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f8e7653d640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e700af4d0 0x7f8e7013e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f8e7653d640 1 -- 172.21.15.16:0/3226049090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e70145a80 msgr2=0x7f8e7009af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f8e7653d640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e70145a80 0x7f8e7009af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f8e7653d640 1 -- 172.21.15.16:0/3226049090 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7009dfd0 con 0x7f8e70072520 2024-09-12T01:33:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.833+0000 7f714dbc9640 1 --2- 172.21.15.16:0/1038335322 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f714806e1e0 0x7f7148154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7f8e7653d640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8e70072520 0x7f8e7013e0a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8e68000f80 tx=0x7f8e68004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7f714e3ca640 1 -- 172.21.15.16:0/1038335322 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f714806e1e0 msgr2=0x7f7148154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7f714e3ca640 1 --2- 172.21.15.16:0/1038335322 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f714806e1e0 0x7f7148154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7f714e3ca640 1 -- 172.21.15.16:0/1038335322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7148071fb0 msgr2=0x7f71481554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7f714e3ca640 1 --2- 172.21.15.16:0/1038335322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7148071fb0 0x7f71481554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7f714e3ca640 1 -- 172.21.15.16:0/1038335322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714815c9a0 con 0x7f714806de00 2024-09-12T01:33:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7f714e3ca640 1 --2- 172.21.15.16:0/1038335322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f714806de00 0x7f714815c040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f714400c7f0 tx=0x7f714400cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7f8e66ffd640 1 -- 172.21.15.16:0/3226049090 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68041020 con 0x7f8e70072520 2024-09-12T01:33:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7f8e66ffd640 1 -- 172.21.15.16:0/3226049090 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e680087b0 con 0x7f8e70072520 2024-09-12T01:33:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7f8e66ffd640 1 -- 172.21.15.16:0/3226049090 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68036040 con 0x7f8e70072520 2024-09-12T01:33:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7f8e7753f640 1 -- 172.21.15.16:0/3226049090 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7013cbd0 con 0x7f8e70072520 2024-09-12T01:33:35.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7fc3cdf5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3c806bfc0 0x7fc3c81f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7ff66e178640 1 -- 172.21.15.16:0/3635048473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650003bb0 msgr2=0x7ff650003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7ff66e178640 1 --2- 172.21.15.16:0/3635048473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650003bb0 0x7ff650003f90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff66000c5b0 tx=0x7ff650005830 comp rx=0 tx=0).stop 2024-09-12T01:33:35.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7ff66e178640 1 -- 172.21.15.16:0/3635048473 shutdown_connections 2024-09-12T01:33:35.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7ff66e178640 1 --2- 172.21.15.16:0/3635048473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650003bb0 0x7ff650003f90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7ff66e178640 1 -- 172.21.15.16:0/3635048473 >> 172.21.15.16:0/3635048473 conn(0x7ff6680b1af0 msgr2=0x7ff6680b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:35.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7f8e7753f640 1 -- 172.21.15.16:0/3226049090 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7013d470 con 0x7f8e70072520 2024-09-12T01:33:35.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7fc3cdf5f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc3c806eca0 0x7fc3c81f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7fc3c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3c806bfc0 0x7fc3c81f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7fc3cdf5f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c815a270 0x7fc3c8150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7fc3cdf5f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c8069dd0 con 0x7fc3c806bfc0 2024-09-12T01:33:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7fc3cdf5f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c815a650 con 0x7fc3c815a270 2024-09-12T01:33:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7fc3cdf5f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c8069f50 con 0x7fc3c806eca0 2024-09-12T01:33:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.834+0000 7ff66e178640 1 -- 172.21.15.16:0/3635048473 shutdown_connections 2024-09-12T01:33:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7ff66e178640 1 -- 172.21.15.16:0/3635048473 wait complete. 2024-09-12T01:33:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7fc3c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c815a270 0x7fc3c8150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7fc3c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c815a270 0x7fc3c8150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45844/0 (socket says 172.21.15.16:45844) 2024-09-12T01:33:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7fc3c7fff640 1 -- 172.21.15.16:0/3340393529 learned_addr learned my addr 172.21.15.16:0/3340393529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7fc3c6ffd640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc3c806eca0 0x7fc3c81f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7f713effd640 1 -- 172.21.15.16:0/1038335322 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144015070 con 0x7f714806de00 2024-09-12T01:33:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7f714f3cc640 1 -- 172.21.15.16:0/1038335322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714815cc80 con 0x7f714806de00 2024-09-12T01:33:35.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7f714f3cc640 1 -- 172.21.15.16:0/1038335322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714815d240 con 0x7f714806de00 2024-09-12T01:33:35.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7f6bc7fff640 1 -- 172.21.15.16:0/1618119498 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6bd0058570 con 0x7f6be00cdde0 2024-09-12T01:33:35.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7f215ffff640 1 -- 172.21.15.16:0/281296491 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217000be60 con 0x7f21780ccf80 2024-09-12T01:33:35.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7f215ffff640 1 -- 172.21.15.16:0/281296491 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2170019070 con 0x7f21780ccf80 2024-09-12T01:33:35.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7f215ffff640 1 -- 172.21.15.16:0/281296491 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217000eea0 con 0x7f21780ccf80 2024-09-12T01:33:35.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7f8e7753f640 1 -- 172.21.15.16:0/3226049090 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e700ad480 con 0x7f8e70072520 2024-09-12T01:33:35.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.836+0000 7f217e03c640 1 -- 172.21.15.16:0/281296491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217809c170 con 0x7f21780ccf80 2024-09-12T01:33:35.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.836+0000 7f217e03c640 1 -- 172.21.15.16:0/281296491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217809c730 con 0x7f21780ccf80 2024-09-12T01:33:35.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.836+0000 7f9beffff640 1 -- 172.21.15.16:0/3912045347 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4040070 con 0x7f9bf80d0b10 2024-09-12T01:33:35.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.836+0000 7f9beffff640 1 -- 172.21.15.16:0/3912045347 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf4033be0 con 0x7f9bf80d0b10 2024-09-12T01:33:35.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.836+0000 7f9beffff640 1 -- 172.21.15.16:0/3912045347 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8001840 con 0x7f9bf80d0b10 2024-09-12T01:33:35.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7f713effd640 1 -- 172.21.15.16:0/1038335322 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7144007a30 con 0x7f714806de00 2024-09-12T01:33:35.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.835+0000 7f713effd640 1 -- 172.21.15.16:0/1038335322 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144007e00 con 0x7f714806de00 2024-09-12T01:33:35.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.836+0000 7f714f3cc640 1 -- 172.21.15.16:0/1038335322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7130005290 con 0x7f714806de00 2024-09-12T01:33:35.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7fc3c6ffd640 1 -- 172.21.15.16:0/3340393529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c815a270 msgr2=0x7fc3c8150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7fc3c6ffd640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c815a270 0x7fc3c8150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7fc3c6ffd640 1 -- 172.21.15.16:0/3340393529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3c806bfc0 msgr2=0x7fc3c81f2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7fc3c6ffd640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3c806bfc0 0x7fc3c81f2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7fc3c6ffd640 1 -- 172.21.15.16:0/3340393529 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c815a9b0 con 0x7fc3c806eca0 2024-09-12T01:33:35.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7fc3c7fff640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c815a270 0x7fc3c8150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:35.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7ff66e178640 1 Processor -- start 2024-09-12T01:33:35.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7f8e66ffd640 1 -- 172.21.15.16:0/3226049090 <== mon.2 v2:172.21.15.33:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e68044020 con 0x7f8e70072520 2024-09-12T01:33:35.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7f217e03c640 1 -- 172.21.15.16:0/281296491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21780af090 con 0x7f21780ccf80 2024-09-12T01:33:35.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7fc3c6ffd640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc3c806eca0 0x7fc3c81f0de0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8073400 tx=0x7fc3b8001b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7f9bfe394640 1 -- 172.21.15.16:0/3912045347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf80d0b10 msgr2=0x7f9bf8072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7f9bfe394640 1 --2- 172.21.15.16:0/3912045347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf80d0b10 0x7f9bf8072f70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8072d30 tx=0x7f9bf4033150 comp rx=0 tx=0).stop 2024-09-12T01:33:35.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7f9bfe394640 1 -- 172.21.15.16:0/3912045347 shutdown_connections 2024-09-12T01:33:35.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7f9bfe394640 1 --2- 172.21.15.16:0/3912045347 >> v2:172.21.15.16:3300/0 conn(0x7f9bf806ee10 0x7f9bf806f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7f9bfe394640 1 --2- 172.21.15.16:0/3912045347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf80d0b10 0x7f9bf8072f70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7f9bfe394640 1 -- 172.21.15.16:0/3912045347 >> 172.21.15.16:0/3912045347 conn(0x7f9bf80707e0 msgr2=0x7f9bf8070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:35.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.837+0000 7ff66e178640 1 -- start start 2024-09-12T01:33:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.838+0000 7f8e66ffd640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8e5406bf00 0x7f8e5406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.838+0000 7f9bfe394640 1 -- 172.21.15.16:0/3912045347 shutdown_connections 2024-09-12T01:33:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.838+0000 7f8e66ffd640 1 -- 172.21.15.16:0/3226049090 <== mon.2 v2:172.21.15.33:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8e680abdb0 con 0x7f8e70072520 2024-09-12T01:33:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.838+0000 7f9bfe394640 1 -- 172.21.15.16:0/3912045347 wait complete. 2024-09-12T01:33:35.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.838+0000 7f713effd640 1 -- 172.21.15.16:0/1038335322 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7144018070 con 0x7f714806de00 2024-09-12T01:33:35.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.838+0000 7f8e75d3c640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8e5406bf00 0x7f8e5406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.838+0000 7f215ffff640 1 -- 172.21.15.16:0/281296491 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2170005d10 con 0x7f21780ccf80 2024-09-12T01:33:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.838+0000 7f713effd640 1 --2- 172.21.15.16:0/1038335322 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f712c06be30 0x7f712c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.839+0000 7f713effd640 1 -- 172.21.15.16:0/1038335322 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f714408ab20 con 0x7f714806de00 2024-09-12T01:33:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.839+0000 7f714dbc9640 1 --2- 172.21.15.16:0/1038335322 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f712c06be30 0x7f712c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.839+0000 7f215ffff640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f216406bf00 0x7f216406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.839+0000 7f8e75d3c640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8e5406bf00 0x7f8e5406e3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8e700b4f50 tx=0x7f8e6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.839+0000 7f215ffff640 1 -- 172.21.15.16:0/281296491 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f217008b9a0 con 0x7f21780ccf80 2024-09-12T01:33:35.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.839+0000 7f714dbc9640 1 --2- 172.21.15.16:0/1038335322 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f712c06be30 0x7f712c06e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7138034000 tx=0x7f713802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.839+0000 7ff66e178640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815bc10 0x7ff6681573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.839+0000 7ff66e178640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6680b42e0 0x7ff66809aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff66e178640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff66809af70 0x7ff66815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff6677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815bc10 0x7ff6681573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.839+0000 7f21777fe640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f216406bf00 0x7f216406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff66e178640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6680b25e0 con 0x7ff66815bc10 2024-09-12T01:33:35.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff66e178640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff66815def0 con 0x7ff6680b42e0 2024-09-12T01:33:35.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff66e178640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff6680b2760 con 0x7ff66809af70 2024-09-12T01:33:35.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7f9bfe394640 1 Processor -- start 2024-09-12T01:33:35.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff6677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815bc10 0x7ff6681573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42428/0 (socket says 172.21.15.16:42428) 2024-09-12T01:33:35.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff6677fe640 1 -- 172.21.15.16:0/2939717370 learned_addr learned my addr 172.21.15.16:0/2939717370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:35.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff667fff640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff66809af70 0x7ff66815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7f9bfe394640 1 -- start start 2024-09-12T01:33:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff666ffd640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6680b42e0 0x7ff66809aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7f21777fe640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f216406bf00 0x7f216406e3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f216803b000 tx=0x7f216800d000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff667fff640 1 -- 172.21.15.16:0/2939717370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6680b42e0 msgr2=0x7ff66809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff667fff640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6680b42e0 0x7ff66809aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff667fff640 1 -- 172.21.15.16:0/2939717370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815bc10 msgr2=0x7ff6681573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff667fff640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815bc10 0x7ff6681573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff667fff640 1 -- 172.21.15.16:0/2939717370 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6680b5960 con 0x7ff66809af70 2024-09-12T01:33:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.840+0000 7ff6677fe640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815bc10 0x7ff6681573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7ff667fff640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff66809af70 0x7ff66815bff0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff66815a910 tx=0x7ff65800ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfe394640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9bf806ee10 0x7f9bf81f3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfe394640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf80d0b10 0x7f9bf80ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfe394640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8144c20 0x7f9bf806cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfe394640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf815ed70 con 0x7f9bf8144c20 2024-09-12T01:33:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfe394640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf815ebf0 con 0x7f9bf80d0b10 2024-09-12T01:33:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfe394640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf80b43f0 con 0x7f9bf806ee10 2024-09-12T01:33:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfdb93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8144c20 0x7f9bf806cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfd392640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9bf806ee10 0x7f9bf81f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfcb91640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf80d0b10 0x7f9bf80ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfdb93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8144c20 0x7f9bf806cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42432/0 (socket says 172.21.15.16:42432) 2024-09-12T01:33:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfdb93640 1 -- 172.21.15.16:0/3901275536 learned_addr learned my addr 172.21.15.16:0/3901275536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfdb93640 1 -- 172.21.15.16:0/3901275536 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9bf806ee10 msgr2=0x7f9bf81f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfdb93640 1 --2- 172.21.15.16:0/3901275536 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9bf806ee10 0x7f9bf81f3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfdb93640 1 -- 172.21.15.16:0/3901275536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf80d0b10 msgr2=0x7f9bf80ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfdb93640 1 --2- 172.21.15.16:0/3901275536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf80d0b10 0x7f9bf80ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.841+0000 7f9bfdb93640 1 -- 172.21.15.16:0/3901275536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf81f3b50 con 0x7f9bf8144c20 2024-09-12T01:33:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.842+0000 7f9bfdb93640 1 --2- 172.21.15.16:0/3901275536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8144c20 0x7f9bf806cb40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000a670 tx=0x7f9bf000ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.842+0000 7f9bedffb640 1 -- 172.21.15.16:0/3901275536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf001a070 con 0x7f9bf8144c20 2024-09-12T01:33:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.842+0000 7f9bfe394640 1 -- 172.21.15.16:0/3901275536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf81f3e30 con 0x7f9bf8144c20 2024-09-12T01:33:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.842+0000 7f9bedffb640 1 -- 172.21.15.16:0/3901275536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf000d800 con 0x7f9bf8144c20 2024-09-12T01:33:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.842+0000 7f9bedffb640 1 -- 172.21.15.16:0/3901275536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf000dbf0 con 0x7f9bf8144c20 2024-09-12T01:33:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.842+0000 7fc3affff640 1 -- 172.21.15.16:0/3340393529 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b80074e0 con 0x7fc3c806eca0 2024-09-12T01:33:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.842+0000 7fc3affff640 1 -- 172.21.15.16:0/3340393529 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b8007680 con 0x7fc3c806eca0 2024-09-12T01:33:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.842+0000 7fc3affff640 1 -- 172.21.15.16:0/3340393529 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8004a20 con 0x7fc3c806eca0 2024-09-12T01:33:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.842+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/3340393529 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c815ac90 con 0x7fc3c806eca0 2024-09-12T01:33:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.843+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/3340393529 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c815d8a0 con 0x7fc3c806eca0 2024-09-12T01:33:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.842+0000 7ff64ffff640 1 -- 172.21.15.16:0/2939717370 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658019070 con 0x7ff66809af70 2024-09-12T01:33:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.842+0000 7ff64ffff640 1 -- 172.21.15.16:0/2939717370 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff658006d60 con 0x7ff66809af70 2024-09-12T01:33:35.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.842+0000 7f9bfe394640 1 -- 172.21.15.16:0/3901275536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf81f3740 con 0x7f9bf8144c20 2024-09-12T01:33:35.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.843+0000 7f8e66ffd640 1 -- 172.21.15.16:0/3226049090 <== mon.2 v2:172.21.15.33:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8e6804b030 con 0x7f8e70072520 2024-09-12T01:33:35.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.844+0000 7ff64ffff640 1 -- 172.21.15.16:0/2939717370 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658003b90 con 0x7ff66809af70 2024-09-12T01:33:35.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.844+0000 7f713effd640 1 -- 172.21.15.16:0/1038335322 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7144058aa0 con 0x7f714806de00 2024-09-12T01:33:35.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.844+0000 7f9bfe394640 1 -- 172.21.15.16:0/3901275536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf8069700 con 0x7f9bf8144c20 2024-09-12T01:33:35.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.844+0000 7ff66e178640 1 -- 172.21.15.16:0/2939717370 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6680b5c40 con 0x7ff66809af70 2024-09-12T01:33:35.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.844+0000 7ff66e178640 1 -- 172.21.15.16:0/2939717370 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6680b3ae0 con 0x7ff66809af70 2024-09-12T01:33:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.845+0000 7fc3affff640 1 -- 172.21.15.16:0/3340393529 <== mon.2 v2:172.21.15.33:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3b8004bc0 con 0x7fc3c806eca0 2024-09-12T01:33:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.846+0000 7fc3affff640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc3b406be30 0x7fc3b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.846+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/3340393529 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a8005290 con 0x7fc3c806eca0 2024-09-12T01:33:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.845+0000 7f9bedffb640 1 -- 172.21.15.16:0/3901275536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bf001d070 con 0x7f9bf8144c20 2024-09-12T01:33:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.846+0000 7f9bedffb640 1 --2- 172.21.15.16:0/3901275536 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.846+0000 7fc3affff640 1 -- 172.21.15.16:0/3340393529 <== mon.2 v2:172.21.15.33:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc3b808a690 con 0x7fc3c806eca0 2024-09-12T01:33:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.846+0000 7f9bedffb640 1 -- 172.21.15.16:0/3901275536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9bf0090060 con 0x7f9bf8144c20 2024-09-12T01:33:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.846+0000 7f9bfd392640 1 --2- 172.21.15.16:0/3901275536 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.846+0000 7ff66e178640 1 -- 172.21.15.16:0/2939717370 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6680cd360 con 0x7ff66809af70 2024-09-12T01:33:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.847+0000 7fc3c77fe640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc3b406be30 0x7fc3b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.847+0000 7ff64ffff640 1 -- 172.21.15.16:0/2939717370 <== mon.2 v2:172.21.15.33:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff65800d740 con 0x7ff66809af70 2024-09-12T01:33:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.847+0000 7fc3c77fe640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc3b406be30 0x7fc3b406e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc035000 tx=0x7fc3bc02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.847+0000 7f9bfd392640 1 --2- 172.21.15.16:0/3901275536 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9bf403d000 tx=0x7f9bf4037000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.848+0000 7f215ffff640 1 -- 172.21.15.16:0/281296491 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2170059ff0 con 0x7f21780ccf80 2024-09-12T01:33:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.848+0000 7ff64ffff640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff65406be60 0x7ff65406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.848+0000 7ff64ffff640 1 -- 172.21.15.16:0/2939717370 <== mon.2 v2:172.21.15.33:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff65808a4b0 con 0x7ff66809af70 2024-09-12T01:33:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.848+0000 7ff6677fe640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff65406be60 0x7ff65406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.849+0000 7ff6677fe640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff65406be60 0x7ff65406e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff660000c00 tx=0x7ff660032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:35.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.851+0000 7fc3affff640 1 -- 172.21.15.16:0/3340393529 <== mon.2 v2:172.21.15.33:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc3b8058d30 con 0x7fc3c806eca0 2024-09-12T01:33:35.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.854+0000 7f9bedffb640 1 -- 172.21.15.16:0/3901275536 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9bf0059800 con 0x7f9bf8144c20 2024-09-12T01:33:35.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:35.857+0000 7ff64ffff640 1 -- 172.21.15.16:0/2939717370 <== mon.2 v2:172.21.15.33:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff658058430 con 0x7ff66809af70 2024-09-12T01:33:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.192+0000 7f160693d640 1 -- 172.21.15.16:0/1211004463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1600072710 con 0x7f16000b26c0 2024-09-12T01:33:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.193+0000 7f15f67fc640 1 -- 172.21.15.16:0/1211004463 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f15fc079c60 con 0x7f16000b26c0 2024-09-12T01:33:36.193 INFO:teuthology.orchestra.run.smithi016.stdout:64424509451 2024-09-12T01:33:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.197+0000 7f15e3fff640 1 -- 172.21.15.16:0/1211004463 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f15e406bf30 msgr2=0x7f15e406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.197+0000 7f15e3fff640 1 --2- 172.21.15.16:0/1211004463 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f15e406bf30 0x7f15e406e3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f15f0006fd0 tx=0x7f15f0008040 comp rx=0 tx=0).stop 2024-09-12T01:33:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.197+0000 7f15e3fff640 1 -- 172.21.15.16:0/1211004463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000b26c0 msgr2=0x7f160009aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.198+0000 7f15e3fff640 1 --2- 172.21.15.16:0/1211004463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000b26c0 0x7f160009aca0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0097f0 tx=0x7f15fc0046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.198+0000 7f15e3fff640 1 -- 172.21.15.16:0/1211004463 shutdown_connections 2024-09-12T01:33:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.198+0000 7f15e3fff640 1 --2- 172.21.15.16:0/1211004463 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f15e406bf30 0x7f15e406e3f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.198+0000 7f15e3fff640 1 --2- 172.21.15.16:0/1211004463 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f160015dea0 0x7f16001fbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.198+0000 7f15e3fff640 1 --2- 172.21.15.16:0/1211004463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000b26c0 0x7f160009aca0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.198+0000 7f15e3fff640 1 --2- 172.21.15.16:0/1211004463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160006deb0 0x7f160009d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.198+0000 7f15e3fff640 1 -- 172.21.15.16:0/1211004463 >> 172.21.15.16:0/1211004463 conn(0x7f16000b1bd0 msgr2=0x7f160015b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.198+0000 7f15e3fff640 1 -- 172.21.15.16:0/1211004463 shutdown_connections 2024-09-12T01:33:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.199+0000 7f15e3fff640 1 -- 172.21.15.16:0/1211004463 wait complete. 2024-09-12T01:33:36.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.6 2024-09-12T01:33:36.215 DEBUG:teuthology.parallel:result is None 2024-09-12T01:33:36.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.233+0000 7f9bfe394640 1 -- 172.21.15.16:0/3901275536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9bf80cd620 con 0x7f9bf8144c20 2024-09-12T01:33:36.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.234+0000 7f9bedffb640 1 -- 172.21.15.16:0/3901275536 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9bf005db00 con 0x7f9bf8144c20 2024-09-12T01:33:36.233 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2024-09-12T01:33:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.236+0000 7f9bfe394640 1 -- 172.21.15.16:0/3901275536 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f9bdc06be30 msgr2=0x7f9bdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.236+0000 7f9bfe394640 1 --2- 172.21.15.16:0/3901275536 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9bf403d000 tx=0x7f9bf4037000 comp rx=0 tx=0).stop 2024-09-12T01:33:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.236+0000 7f9bfe394640 1 -- 172.21.15.16:0/3901275536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8144c20 msgr2=0x7f9bf806cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.236+0000 7f9bfe394640 1 --2- 172.21.15.16:0/3901275536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8144c20 0x7f9bf806cb40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000a670 tx=0x7f9bf000ab30 comp rx=0 tx=0).stop 2024-09-12T01:33:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.236+0000 7f9bfe394640 1 -- 172.21.15.16:0/3901275536 shutdown_connections 2024-09-12T01:33:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.236+0000 7f9bfe394640 1 --2- 172.21.15.16:0/3901275536 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.236+0000 7f9bfe394640 1 --2- 172.21.15.16:0/3901275536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8144c20 0x7f9bf806cb40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.236+0000 7f9bfe394640 1 --2- 172.21.15.16:0/3901275536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf80d0b10 0x7f9bf80ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.236+0000 7f9bfe394640 1 --2- 172.21.15.16:0/3901275536 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9bf806ee10 0x7f9bf81f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.236+0000 7f9bfe394640 1 -- 172.21.15.16:0/3901275536 >> 172.21.15.16:0/3901275536 conn(0x7f9bf80707e0 msgr2=0x7f9bf8143660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.237+0000 7f9bfe394640 1 -- 172.21.15.16:0/3901275536 shutdown_connections 2024-09-12T01:33:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.237+0000 7f9bfe394640 1 -- 172.21.15.16:0/3901275536 wait complete. 2024-09-12T01:33:36.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-09-12T01:33:36.248 DEBUG:teuthology.parallel:result is None 2024-09-12T01:33:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.255+0000 7f6be6201640 1 -- 172.21.15.16:0/1618119498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f6be01625b0 con 0x7f6be00cdde0 2024-09-12T01:33:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.255+0000 7f6bc7fff640 1 -- 172.21.15.16:0/1618119498 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6bd005c870 con 0x7f6be00cdde0 2024-09-12T01:33:36.255 INFO:teuthology.orchestra.run.smithi016.stdout:55834574858 2024-09-12T01:33:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.257+0000 7f714f3cc640 1 -- 172.21.15.16:0/1038335322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7130005580 con 0x7f714806de00 2024-09-12T01:33:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f6be6201640 1 -- 172.21.15.16:0/1618119498 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6bcc06bf00 msgr2=0x7f6bcc06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f6be6201640 1 --2- 172.21.15.16:0/1618119498 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6bcc06bf00 0x7f6bcc06e3c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6bd802a6e0 tx=0x7f6bd802f000 comp rx=0 tx=0).stop 2024-09-12T01:33:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f6be6201640 1 -- 172.21.15.16:0/1618119498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be00cdde0 msgr2=0x7f6be00cbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f6be6201640 1 --2- 172.21.15.16:0/1618119498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be00cdde0 0x7f6be00cbba0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000c390 tx=0x7f6bd000c850 comp rx=0 tx=0).stop 2024-09-12T01:33:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f6be6201640 1 -- 172.21.15.16:0/1618119498 shutdown_connections 2024-09-12T01:33:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f6be6201640 1 --2- 172.21.15.16:0/1618119498 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6bcc06bf00 0x7f6bcc06e3c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f6be6201640 1 --2- 172.21.15.16:0/1618119498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be00ce1c0 0x7f6be00cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f6be6201640 1 --2- 172.21.15.16:0/1618119498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be00cdde0 0x7f6be00cbba0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f6be6201640 1 --2- 172.21.15.16:0/1618119498 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6be006f750 0x7f6be009c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f6be6201640 1 -- 172.21.15.16:0/1618119498 >> 172.21.15.16:0/1618119498 conn(0x7f6be00b1bd0 msgr2=0x7f6be00b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f713effd640 1 -- 172.21.15.16:0/1038335322 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f714405cda0 con 0x7f714806de00 2024-09-12T01:33:36.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f6be6201640 1 -- 172.21.15.16:0/1618119498 shutdown_connections 2024-09-12T01:33:36.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f6be6201640 1 -- 172.21.15.16:0/1618119498 wait complete. 2024-09-12T01:33:36.259 INFO:teuthology.orchestra.run.smithi016.stdout:60129542153 2024-09-12T01:33:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.258+0000 7f8e7753f640 1 -- 172.21.15.16:0/3226049090 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8e7009acb0 con 0x7f8e70072520 2024-09-12T01:33:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.259+0000 7f8e66ffd640 1 -- 172.21.15.16:0/3226049090 <== mon.2 v2:172.21.15.33:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8e6802ae20 con 0x7f8e70072520 2024-09-12T01:33:36.260 INFO:teuthology.orchestra.run.smithi016.stdout:60129542154 2024-09-12T01:33:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.260+0000 7f714f3cc640 1 -- 172.21.15.16:0/1038335322 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f712c06be30 msgr2=0x7f712c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.260+0000 7f714f3cc640 1 --2- 172.21.15.16:0/1038335322 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f712c06be30 0x7f712c06e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7138034000 tx=0x7f713802e000 comp rx=0 tx=0).stop 2024-09-12T01:33:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.260+0000 7f714f3cc640 1 -- 172.21.15.16:0/1038335322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f714806de00 msgr2=0x7f714815c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.260+0000 7f714f3cc640 1 --2- 172.21.15.16:0/1038335322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f714806de00 0x7f714815c040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f714400c7f0 tx=0x7f714400cbf0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.260+0000 7f714f3cc640 1 -- 172.21.15.16:0/1038335322 shutdown_connections 2024-09-12T01:33:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.260+0000 7f714f3cc640 1 --2- 172.21.15.16:0/1038335322 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f712c06be30 0x7f712c06e2f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.260+0000 7f714f3cc640 1 --2- 172.21.15.16:0/1038335322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7148071fb0 0x7f71481554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.260+0000 7f714f3cc640 1 --2- 172.21.15.16:0/1038335322 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f714806e1e0 0x7f7148154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.260+0000 7f714f3cc640 1 --2- 172.21.15.16:0/1038335322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f714806de00 0x7f714815c040 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.260+0000 7f714f3cc640 1 -- 172.21.15.16:0/1038335322 >> 172.21.15.16:0/1038335322 conn(0x7f71480b1bd0 msgr2=0x7f7148160660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.261+0000 7f714f3cc640 1 -- 172.21.15.16:0/1038335322 shutdown_connections 2024-09-12T01:33:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.261+0000 7f714f3cc640 1 -- 172.21.15.16:0/1038335322 wait complete. 2024-09-12T01:33:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.262+0000 7f8e7753f640 1 -- 172.21.15.16:0/3226049090 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8e5406bf00 msgr2=0x7f8e5406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.262+0000 7f8e7753f640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8e5406bf00 0x7f8e5406e3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8e700b4f50 tx=0x7f8e6c008040 comp rx=0 tx=0).stop 2024-09-12T01:33:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.262+0000 7f8e7753f640 1 -- 172.21.15.16:0/3226049090 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8e70072520 msgr2=0x7f8e7013e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.262+0000 7f8e7753f640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8e70072520 0x7f8e7013e0a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8e68000f80 tx=0x7f8e68004a40 comp rx=0 tx=0).stop 2024-09-12T01:33:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.262+0000 7f8e7753f640 1 -- 172.21.15.16:0/3226049090 shutdown_connections 2024-09-12T01:33:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.262+0000 7f8e7753f640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8e5406bf00 0x7f8e5406e3c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.262+0000 7f8e7753f640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e70145a80 0x7f8e7009af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.262+0000 7f8e7753f640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e700af4d0 0x7f8e7013e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.262+0000 7f8e7753f640 1 --2- 172.21.15.16:0/3226049090 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8e70072520 0x7f8e7013e0a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.262+0000 7f8e7753f640 1 -- 172.21.15.16:0/3226049090 >> 172.21.15.16:0/3226049090 conn(0x7f8e700b1bd0 msgr2=0x7f8e7015e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.262+0000 7f8e7753f640 1 -- 172.21.15.16:0/3226049090 shutdown_connections 2024-09-12T01:33:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.262+0000 7f8e7753f640 1 -- 172.21.15.16:0/3226049090 wait complete. 2024-09-12T01:33:36.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.265+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/3340393529 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc3a8005580 con 0x7fc3c806eca0 2024-09-12T01:33:36.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.266+0000 7fc3affff640 1 -- 172.21.15.16:0/3340393529 <== mon.2 v2:172.21.15.33:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc3b8058630 con 0x7fc3c806eca0 2024-09-12T01:33:36.266 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2024-09-12T01:33:36.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.267+0000 7f217e03c640 1 -- 172.21.15.16:0/281296491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f21780adac0 con 0x7f21780ccf80 2024-09-12T01:33:36.267 INFO:teuthology.orchestra.run.smithi016.stdout:64424509451 2024-09-12T01:33:36.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.268+0000 7f215ffff640 1 -- 172.21.15.16:0/281296491 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f21700598f0 con 0x7f21780ccf80 2024-09-12T01:33:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.269+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/3340393529 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc3b406be30 msgr2=0x7fc3b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.269+0000 7fc3cdf5f640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc3b406be30 0x7fc3b406e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc035000 tx=0x7fc3bc02f000 comp rx=0 tx=0).stop 2024-09-12T01:33:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.269+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/3340393529 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc3c806eca0 msgr2=0x7fc3c81f0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.269+0000 7fc3cdf5f640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc3c806eca0 0x7fc3c81f0de0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8073400 tx=0x7fc3b8001b80 comp rx=0 tx=0).stop 2024-09-12T01:33:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.269+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/3340393529 shutdown_connections 2024-09-12T01:33:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.269+0000 7fc3cdf5f640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc3b406be30 0x7fc3b406e2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.269+0000 7fc3cdf5f640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c815a270 0x7fc3c8150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.269+0000 7fc3cdf5f640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc3c806eca0 0x7fc3c81f0de0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.269+0000 7fc3cdf5f640 1 --2- 172.21.15.16:0/3340393529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3c806bfc0 0x7fc3c81f2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.269+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/3340393529 >> 172.21.15.16:0/3340393529 conn(0x7fc3c80b2510 msgr2=0x7fc3c80b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.269+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/3340393529 shutdown_connections 2024-09-12T01:33:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.269+0000 7fc3cdf5f640 1 -- 172.21.15.16:0/3340393529 wait complete. 2024-09-12T01:33:36.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.270+0000 7ff66e178640 1 -- 172.21.15.16:0/2939717370 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff6680cefb0 con 0x7ff66809af70 2024-09-12T01:33:36.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.271+0000 7ff64ffff640 1 -- 172.21.15.16:0/2939717370 <== mon.2 v2:172.21.15.33:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff65805c730 con 0x7ff66809af70 2024-09-12T01:33:36.270 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2024-09-12T01:33:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.271+0000 7f217e03c640 1 -- 172.21.15.16:0/281296491 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f216406bf00 msgr2=0x7f216406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.271+0000 7f217e03c640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f216406bf00 0x7f216406e3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f216803b000 tx=0x7f216800d000 comp rx=0 tx=0).stop 2024-09-12T01:33:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.271+0000 7f217e03c640 1 -- 172.21.15.16:0/281296491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21780ccf80 msgr2=0x7f21780b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.271+0000 7f217e03c640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21780ccf80 0x7f21780b3b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f21700070b0 tx=0x7f217000e4f0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.271+0000 7f217e03c640 1 -- 172.21.15.16:0/281296491 shutdown_connections 2024-09-12T01:33:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.271+0000 7f217e03c640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f216406bf00 0x7f216406e3c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.271+0000 7f217e03c640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f21780cd360 0x7f21781f0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.271+0000 7f217e03c640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21780ccf80 0x7f21780b3b00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.271+0000 7f217e03c640 1 --2- 172.21.15.16:0/281296491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21780af5d0 0x7f21780c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.271+0000 7f217e03c640 1 -- 172.21.15.16:0/281296491 >> 172.21.15.16:0/281296491 conn(0x7f21780b1af0 msgr2=0x7f21780acdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.272+0000 7f217e03c640 1 -- 172.21.15.16:0/281296491 shutdown_connections 2024-09-12T01:33:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.272+0000 7f217e03c640 1 -- 172.21.15.16:0/281296491 wait complete. 2024-09-12T01:33:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.273+0000 7ff66e178640 1 -- 172.21.15.16:0/2939717370 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff65406be60 msgr2=0x7ff65406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.274+0000 7ff66e178640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff65406be60 0x7ff65406e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff660000c00 tx=0x7ff660032000 comp rx=0 tx=0).stop 2024-09-12T01:33:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.274+0000 7ff66e178640 1 -- 172.21.15.16:0/2939717370 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff66809af70 msgr2=0x7ff66815bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.274+0000 7ff66e178640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff66809af70 0x7ff66815bff0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff66815a910 tx=0x7ff65800ff70 comp rx=0 tx=0).stop 2024-09-12T01:33:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.274+0000 7ff66e178640 1 -- 172.21.15.16:0/2939717370 shutdown_connections 2024-09-12T01:33:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.274+0000 7ff66e178640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff65406be60 0x7ff65406e320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.274+0000 7ff66e178640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff66809af70 0x7ff66815bff0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.274+0000 7ff66e178640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6680b42e0 0x7ff66809aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.274+0000 7ff66e178640 1 --2- 172.21.15.16:0/2939717370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815bc10 0x7ff6681573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.274+0000 7ff66e178640 1 -- 172.21.15.16:0/2939717370 >> 172.21.15.16:0/2939717370 conn(0x7ff6680b1af0 msgr2=0x7ff668158090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.274+0000 7ff66e178640 1 -- 172.21.15.16:0/2939717370 shutdown_connections 2024-09-12T01:33:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.274+0000 7ff66e178640 1 -- 172.21.15.16:0/2939717370 wait complete. 2024-09-12T01:33:36.276 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-09-12T01:33:36.277 DEBUG:teuthology.parallel:result is None 2024-09-12T01:33:36.280 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.3 2024-09-12T01:33:36.280 DEBUG:teuthology.parallel:result is None 2024-09-12T01:33:36.280 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.1 2024-09-12T01:33:36.281 DEBUG:teuthology.parallel:result is None 2024-09-12T01:33:36.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.7 2024-09-12T01:33:36.286 DEBUG:teuthology.parallel:result is None 2024-09-12T01:33:36.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.4 2024-09-12T01:33:36.290 DEBUG:teuthology.parallel:result is None 2024-09-12T01:33:36.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.5 2024-09-12T01:33:36.291 DEBUG:teuthology.parallel:result is None 2024-09-12T01:33:36.291 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T01:33:36.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:33:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.435+0000 7fd221ce4640 1 Processor -- start 2024-09-12T01:33:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.435+0000 7fd221ce4640 1 -- start start 2024-09-12T01:33:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.435+0000 7fd221ce4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd21c15e5a0 0x7fd21c0d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.435+0000 7fd221ce4640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd21c0d2d70 0x7fd21c0d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.435+0000 7fd221ce4640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd21c0d16b0 con 0x7fd21c0d1200 2024-09-12T01:33:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.435+0000 7fd221ce4640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fd21c15e980 con 0x7fd21c0d2d70 2024-09-12T01:33:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.435+0000 7fd221ce4640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c0d3ee0 con 0x7fd21c15e5a0 2024-09-12T01:33:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.436+0000 7fd21b7fe640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd21c15e5a0 0x7fd21c0d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.436+0000 7fd21affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd21c0d2d70 0x7fd21c0d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.436+0000 7fd21bfff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd21c0d1200 0x7fd21c0b5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:44526/0 (socket says 172.21.15.16:44526) 2024-09-12T01:33:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.436+0000 7fd21bfff640 1 -- 172.21.15.16:0/3086142556 learned_addr learned my addr 172.21.15.16:0/3086142556 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.436+0000 7fd21a7fc640 1 -- 172.21.15.16:0/3086142556 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2475090450 0 0) 0x7fd214001490 con 0x7fd21c0d1200 2024-09-12T01:33:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.436+0000 7fd21a7fc640 1 -- 172.21.15.16:0/3086142556 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd204003520 con 0x7fd21c0d1200 2024-09-12T01:33:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.436+0000 7fd21affd640 1 -- 172.21.15.16:0/3086142556 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd21c15e5a0 msgr2=0x7fd21c0d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.436+0000 7fd21affd640 1 --2- 172.21.15.16:0/3086142556 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd21c15e5a0 0x7fd21c0d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.436+0000 7fd21affd640 1 -- 172.21.15.16:0/3086142556 >> v1:172.21.15.33:6789/0 conn(0x7fd21c0d1200 legacy=0x7fd21c0b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.436+0000 7fd21affd640 1 -- 172.21.15.16:0/3086142556 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c0d32d0 con 0x7fd21c0d2d70 2024-09-12T01:33:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.436+0000 7fd21b7fe640 1 --2- 172.21.15.16:0/3086142556 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd21c15e5a0 0x7fd21c0d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:36.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.437+0000 7fd21affd640 1 --2- 172.21.15.16:0/3086142556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd21c0d2d70 0x7fd21c0d0cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd2100097a0 tx=0x7fd21002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73a4e2564d7d6f5f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:36.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.437+0000 7fd21a7fc640 1 -- 172.21.15.16:0/3086142556 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210037070 con 0x7fd21c0d2d70 2024-09-12T01:33:36.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.437+0000 7fd21a7fc640 1 -- 172.21.15.16:0/3086142556 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd210032440 con 0x7fd21c0d2d70 2024-09-12T01:33:36.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.437+0000 7fd21a7fc640 1 -- 172.21.15.16:0/3086142556 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2100327f0 con 0x7fd21c0d2d70 2024-09-12T01:33:36.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.438+0000 7fd221ce4640 1 -- 172.21.15.16:0/3086142556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd21c0d2d70 msgr2=0x7fd21c0d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.438+0000 7fd221ce4640 1 --2- 172.21.15.16:0/3086142556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd21c0d2d70 0x7fd21c0d0cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd2100097a0 tx=0x7fd21002a180 comp rx=0 tx=0).stop 2024-09-12T01:33:36.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.438+0000 7fd221ce4640 1 -- 172.21.15.16:0/3086142556 shutdown_connections 2024-09-12T01:33:36.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.438+0000 7fd221ce4640 1 --2- 172.21.15.16:0/3086142556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd21c0d2d70 0x7fd21c0d0cc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.438+0000 7fd221ce4640 1 --2- 172.21.15.16:0/3086142556 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd21c15e5a0 0x7fd21c0d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.438+0000 7fd221ce4640 1 -- 172.21.15.16:0/3086142556 >> 172.21.15.16:0/3086142556 conn(0x7fd21c06d1c0 msgr2=0x7fd21c073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.438+0000 7fd221ce4640 1 -- 172.21.15.16:0/3086142556 shutdown_connections 2024-09-12T01:33:36.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.438+0000 7fd221ce4640 1 -- 172.21.15.16:0/3086142556 wait complete. 2024-09-12T01:33:36.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.439+0000 7fd221ce4640 1 Processor -- start 2024-09-12T01:33:36.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.439+0000 7fd221ce4640 1 -- start start 2024-09-12T01:33:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd221ce4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd21c0d1200 0x7fd21c0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd221ce4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd21c0d2d70 0x7fd21c1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd221ce4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd21c15e5a0 0x7fd21c0ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd221ce4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c0d3b70 con 0x7fd21c0d1200 2024-09-12T01:33:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd221ce4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c0d3890 con 0x7fd21c15e5a0 2024-09-12T01:33:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd221ce4640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c0d22f0 con 0x7fd21c0d2d70 2024-09-12T01:33:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd21b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd21c0d1200 0x7fd21c0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd21b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd21c0d1200 0x7fd21c0cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60384/0 (socket says 172.21.15.16:60384) 2024-09-12T01:33:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd21b7fe640 1 -- 172.21.15.16:0/738223471 learned_addr learned my addr 172.21.15.16:0/738223471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd21bfff640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd21c15e5a0 0x7fd21c0ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd21affd640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd21c0d2d70 0x7fd21c1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd21b7fe640 1 -- 172.21.15.16:0/738223471 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd21c0d2d70 msgr2=0x7fd21c1587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd21b7fe640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd21c0d2d70 0x7fd21c1587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd21b7fe640 1 -- 172.21.15.16:0/738223471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd21c15e5a0 msgr2=0x7fd21c0ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd21b7fe640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd21c15e5a0 0x7fd21c0ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.440+0000 7fd21b7fe640 1 -- 172.21.15.16:0/738223471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c0cc750 con 0x7fd21c0d1200 2024-09-12T01:33:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.441+0000 7fd21b7fe640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd21c0d1200 0x7fd21c0cda90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00e960 tx=0x7fd20c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.441+0000 7fd203fff640 1 -- 172.21.15.16:0/738223471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c017070 con 0x7fd21c0d1200 2024-09-12T01:33:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.441+0000 7fd203fff640 1 -- 172.21.15.16:0/738223471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20c00ca00 con 0x7fd21c0d1200 2024-09-12T01:33:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.441+0000 7fd203fff640 1 -- 172.21.15.16:0/738223471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c00cdd0 con 0x7fd21c0d1200 2024-09-12T01:33:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.441+0000 7fd221ce4640 1 -- 172.21.15.16:0/738223471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21c0cc9f0 con 0x7fd21c0d1200 2024-09-12T01:33:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.441+0000 7fd221ce4640 1 -- 172.21.15.16:0/738223471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21c0ccfb0 con 0x7fd21c0d1200 2024-09-12T01:33:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.442+0000 7fd221ce4640 1 -- 172.21.15.16:0/738223471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd21c0d0cc0 con 0x7fd21c0d1200 2024-09-12T01:33:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.442+0000 7fd203fff640 1 -- 172.21.15.16:0/738223471 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd20c01a070 con 0x7fd21c0d1200 2024-09-12T01:33:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.443+0000 7fd203fff640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd20806be30 0x7fd20806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.443+0000 7fd203fff640 1 -- 172.21.15.16:0/738223471 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd20c08b4b0 con 0x7fd21c0d1200 2024-09-12T01:33:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.443+0000 7fd21affd640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd20806be30 0x7fd20806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.443+0000 7fd21affd640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd20806be30 0x7fd20806e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd210034000 tx=0x7fd21002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.447+0000 7fd203fff640 1 -- 172.21.15.16:0/738223471 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd20c059aa0 con 0x7fd21c0d1200 2024-09-12T01:33:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.622+0000 7fd221ce4640 1 -- 172.21.15.16:0/738223471 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd21c158d30 con 0x7fd20806be30 2024-09-12T01:33:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.630+0000 7fd203fff640 1 -- 172.21.15.16:0/738223471 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fd21c158d30 con 0x7fd20806be30 2024-09-12T01:33:36.634 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:33:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.634+0000 7fd221ce4640 1 -- 172.21.15.16:0/738223471 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd20806be30 msgr2=0x7fd20806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.634+0000 7fd221ce4640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd20806be30 0x7fd20806e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd210034000 tx=0x7fd21002e000 comp rx=0 tx=0).stop 2024-09-12T01:33:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.634+0000 7fd221ce4640 1 -- 172.21.15.16:0/738223471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd21c0d1200 msgr2=0x7fd21c0cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.634+0000 7fd221ce4640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd21c0d1200 0x7fd21c0cda90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00e960 tx=0x7fd20c00ee20 comp rx=0 tx=0).stop 2024-09-12T01:33:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.634+0000 7fd221ce4640 1 -- 172.21.15.16:0/738223471 shutdown_connections 2024-09-12T01:33:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.634+0000 7fd221ce4640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd20806be30 0x7fd20806e2f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.634+0000 7fd221ce4640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd21c15e5a0 0x7fd21c0ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.634+0000 7fd221ce4640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd21c0d2d70 0x7fd21c1587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.634+0000 7fd221ce4640 1 --2- 172.21.15.16:0/738223471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd21c0d1200 0x7fd21c0cda90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.634+0000 7fd221ce4640 1 -- 172.21.15.16:0/738223471 >> 172.21.15.16:0/738223471 conn(0x7fd21c06d1c0 msgr2=0x7fd21c0b92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.634+0000 7fd221ce4640 1 -- 172.21.15.16:0/738223471 shutdown_connections 2024-09-12T01:33:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.634+0000 7fd221ce4640 1 -- 172.21.15.16:0/738223471 wait complete. 2024-09-12T01:33:36.645 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-12T01:33:36.475954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236204,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738516,"statfs":{"total":773094113280,"available":772852240384,"internally_reserved":0,"allocated":3416064,"data_stored":2069540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003204"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413650+0000","last_change":"2024-09-12T01:33:11.148986+0000","last_active":"2024-09-12T01:33:19.413650+0000","last_peered":"2024-09-12T01:33:19.413650+0000","last_clean":"2024-09-12T01:33:19.413650+0000","last_became_active":"2024-09-12T01:33:11.146323+0000","last_became_peered":"2024-09-12T01:33:11.146323+0000","last_unstale":"2024-09-12T01:33:19.413650+0000","last_undegraded":"2024-09-12T01:33:19.413650+0000","last_fullsized":"2024-09-12T01:33:19.413650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:54.118009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217034+0000","last_change":"2024-09-12T01:33:12.183084+0000","last_active":"2024-09-12T01:33:18.217034+0000","last_peered":"2024-09-12T01:33:18.217034+0000","last_clean":"2024-09-12T01:33:18.217034+0000","last_became_active":"2024-09-12T01:33:12.182286+0000","last_became_peered":"2024-09-12T01:33:12.182286+0000","last_unstale":"2024-09-12T01:33:18.217034+0000","last_undegraded":"2024-09-12T01:33:18.217034+0000","last_fullsized":"2024-09-12T01:33:18.217034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:46.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216567+0000","last_change":"2024-09-12T01:33:16.593060+0000","last_active":"2024-09-12T01:33:18.216567+0000","last_peered":"2024-09-12T01:33:18.216567+0000","last_clean":"2024-09-12T01:33:18.216567+0000","last_became_active":"2024-09-12T01:33:16.592584+0000","last_became_peered":"2024-09-12T01:33:16.592584+0000","last_unstale":"2024-09-12T01:33:18.216567+0000","last_undegraded":"2024-09-12T01:33:18.216567+0000","last_fullsized":"2024-09-12T01:33:18.216567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:57.525309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217133+0000","last_change":"2024-09-12T01:33:11.142022+0000","last_active":"2024-09-12T01:33:18.217133+0000","last_peered":"2024-09-12T01:33:18.217133+0000","last_clean":"2024-09-12T01:33:18.217133+0000","last_became_active":"2024-09-12T01:33:11.141709+0000","last_became_peered":"2024-09-12T01:33:11.141709+0000","last_unstale":"2024-09-12T01:33:18.217133+0000","last_undegraded":"2024-09-12T01:33:18.217133+0000","last_fullsized":"2024-09-12T01:33:18.217133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:04.872785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413419+0000","last_change":"2024-09-12T01:33:12.176163+0000","last_active":"2024-09-12T01:33:19.413419+0000","last_peered":"2024-09-12T01:33:19.413419+0000","last_clean":"2024-09-12T01:33:19.413419+0000","last_became_active":"2024-09-12T01:33:12.175795+0000","last_became_peered":"2024-09-12T01:33:12.175795+0000","last_unstale":"2024-09-12T01:33:19.413419+0000","last_undegraded":"2024-09-12T01:33:19.413419+0000","last_fullsized":"2024-09-12T01:33:19.413419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:29.557033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217872+0000","last_change":"2024-09-12T01:33:16.202792+0000","last_active":"2024-09-12T01:33:18.217872+0000","last_peered":"2024-09-12T01:33:18.217872+0000","last_clean":"2024-09-12T01:33:18.217872+0000","last_became_active":"2024-09-12T01:33:16.202352+0000","last_became_peered":"2024-09-12T01:33:16.202352+0000","last_unstale":"2024-09-12T01:33:18.217872+0000","last_undegraded":"2024-09-12T01:33:18.217872+0000","last_fullsized":"2024-09-12T01:33:18.217872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:39.155565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487520+0000","last_change":"2024-09-12T01:33:11.149265+0000","last_active":"2024-09-12T01:33:19.487520+0000","last_peered":"2024-09-12T01:33:19.487520+0000","last_clean":"2024-09-12T01:33:19.487520+0000","last_became_active":"2024-09-12T01:33:11.148975+0000","last_became_peered":"2024-09-12T01:33:11.148975+0000","last_unstale":"2024-09-12T01:33:19.487520+0000","last_undegraded":"2024-09-12T01:33:19.487520+0000","last_fullsized":"2024-09-12T01:33:19.487520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:49.680053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839630+0000","last_change":"2024-09-12T01:33:12.175597+0000","last_active":"2024-09-12T01:33:18.839630+0000","last_peered":"2024-09-12T01:33:18.839630+0000","last_clean":"2024-09-12T01:33:18.839630+0000","last_became_active":"2024-09-12T01:33:12.175271+0000","last_became_peered":"2024-09-12T01:33:12.175271+0000","last_unstale":"2024-09-12T01:33:18.839630+0000","last_undegraded":"2024-09-12T01:33:18.839630+0000","last_fullsized":"2024-09-12T01:33:18.839630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:24.363602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218697+0000","last_change":"2024-09-12T01:33:16.200812+0000","last_active":"2024-09-12T01:33:18.218697+0000","last_peered":"2024-09-12T01:33:18.218697+0000","last_clean":"2024-09-12T01:33:18.218697+0000","last_became_active":"2024-09-12T01:33:16.200029+0000","last_became_peered":"2024-09-12T01:33:16.200029+0000","last_unstale":"2024-09-12T01:33:18.218697+0000","last_undegraded":"2024-09-12T01:33:18.218697+0000","last_fullsized":"2024-09-12T01:33:18.218697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:19.443781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485752+0000","last_change":"2024-09-12T01:33:11.153223+0000","last_active":"2024-09-12T01:33:19.485752+0000","last_peered":"2024-09-12T01:33:19.485752+0000","last_clean":"2024-09-12T01:33:19.485752+0000","last_became_active":"2024-09-12T01:33:11.152938+0000","last_became_peered":"2024-09-12T01:33:11.152938+0000","last_unstale":"2024-09-12T01:33:19.485752+0000","last_undegraded":"2024-09-12T01:33:19.485752+0000","last_fullsized":"2024-09-12T01:33:19.485752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839743+0000","last_change":"2024-09-12T01:33:12.178087+0000","last_active":"2024-09-12T01:33:18.839743+0000","last_peered":"2024-09-12T01:33:18.839743+0000","last_clean":"2024-09-12T01:33:18.839743+0000","last_became_active":"2024-09-12T01:33:12.176556+0000","last_became_peered":"2024-09-12T01:33:12.176556+0000","last_unstale":"2024-09-12T01:33:18.839743+0000","last_undegraded":"2024-09-12T01:33:18.839743+0000","last_fullsized":"2024-09-12T01:33:18.839743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:48.292206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218149+0000","last_change":"2024-09-12T01:33:16.193585+0000","last_active":"2024-09-12T01:33:18.218149+0000","last_peered":"2024-09-12T01:33:18.218149+0000","last_clean":"2024-09-12T01:33:18.218149+0000","last_became_active":"2024-09-12T01:33:16.193050+0000","last_became_peered":"2024-09-12T01:33:16.193050+0000","last_unstale":"2024-09-12T01:33:18.218149+0000","last_undegraded":"2024-09-12T01:33:18.218149+0000","last_fullsized":"2024-09-12T01:33:18.218149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:40.563121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486654+0000","last_change":"2024-09-12T01:33:12.178078+0000","last_active":"2024-09-12T01:33:19.486654+0000","last_peered":"2024-09-12T01:33:19.486654+0000","last_clean":"2024-09-12T01:33:19.486654+0000","last_became_active":"2024-09-12T01:33:12.177806+0000","last_became_peered":"2024-09-12T01:33:12.177806+0000","last_unstale":"2024-09-12T01:33:19.486654+0000","last_undegraded":"2024-09-12T01:33:19.486654+0000","last_fullsized":"2024-09-12T01:33:19.486654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:25.007983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485717+0000","last_change":"2024-09-12T01:33:11.149750+0000","last_active":"2024-09-12T01:33:19.485717+0000","last_peered":"2024-09-12T01:33:19.485717+0000","last_clean":"2024-09-12T01:33:19.485717+0000","last_became_active":"2024-09-12T01:33:11.149442+0000","last_became_peered":"2024-09-12T01:33:11.149442+0000","last_unstale":"2024-09-12T01:33:19.485717+0000","last_undegraded":"2024-09-12T01:33:19.485717+0000","last_fullsized":"2024-09-12T01:33:19.485717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218830+0000","last_change":"2024-09-12T01:33:16.202498+0000","last_active":"2024-09-12T01:33:18.218830+0000","last_peered":"2024-09-12T01:33:18.218830+0000","last_clean":"2024-09-12T01:33:18.218830+0000","last_became_active":"2024-09-12T01:33:16.200800+0000","last_became_peered":"2024-09-12T01:33:16.200800+0000","last_unstale":"2024-09-12T01:33:18.218830+0000","last_undegraded":"2024-09-12T01:33:18.218830+0000","last_fullsized":"2024-09-12T01:33:18.218830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:53.648825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216927+0000","last_change":"2024-09-12T01:33:12.169842+0000","last_active":"2024-09-12T01:33:18.216927+0000","last_peered":"2024-09-12T01:33:18.216927+0000","last_clean":"2024-09-12T01:33:18.216927+0000","last_became_active":"2024-09-12T01:33:12.169524+0000","last_became_peered":"2024-09-12T01:33:12.169524+0000","last_unstale":"2024-09-12T01:33:18.216927+0000","last_undegraded":"2024-09-12T01:33:18.216927+0000","last_fullsized":"2024-09-12T01:33:18.216927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:38.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485299+0000","last_change":"2024-09-12T01:33:11.147221+0000","last_active":"2024-09-12T01:33:19.485299+0000","last_peered":"2024-09-12T01:33:19.485299+0000","last_clean":"2024-09-12T01:33:19.485299+0000","last_became_active":"2024-09-12T01:33:11.146928+0000","last_became_peered":"2024-09-12T01:33:11.146928+0000","last_unstale":"2024-09-12T01:33:19.485299+0000","last_undegraded":"2024-09-12T01:33:19.485299+0000","last_fullsized":"2024-09-12T01:33:19.485299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:22.885518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218288+0000","last_change":"2024-09-12T01:33:16.196589+0000","last_active":"2024-09-12T01:33:18.218288+0000","last_peered":"2024-09-12T01:33:18.218288+0000","last_clean":"2024-09-12T01:33:18.218288+0000","last_became_active":"2024-09-12T01:33:16.196092+0000","last_became_peered":"2024-09-12T01:33:16.196092+0000","last_unstale":"2024-09-12T01:33:18.218288+0000","last_undegraded":"2024-09-12T01:33:18.218288+0000","last_fullsized":"2024-09-12T01:33:18.218288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:28.321407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216715+0000","last_change":"2024-09-12T01:33:12.165178+0000","last_active":"2024-09-12T01:33:18.216715+0000","last_peered":"2024-09-12T01:33:18.216715+0000","last_clean":"2024-09-12T01:33:18.216715+0000","last_became_active":"2024-09-12T01:33:12.162823+0000","last_became_peered":"2024-09-12T01:33:12.162823+0000","last_unstale":"2024-09-12T01:33:18.216715+0000","last_undegraded":"2024-09-12T01:33:18.216715+0000","last_fullsized":"2024-09-12T01:33:18.216715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:54.679349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485555+0000","last_change":"2024-09-12T01:33:11.145358+0000","last_active":"2024-09-12T01:33:19.485555+0000","last_peered":"2024-09-12T01:33:19.485555+0000","last_clean":"2024-09-12T01:33:19.485555+0000","last_became_active":"2024-09-12T01:33:11.145035+0000","last_became_peered":"2024-09-12T01:33:11.145035+0000","last_unstale":"2024-09-12T01:33:19.485555+0000","last_undegraded":"2024-09-12T01:33:19.485555+0000","last_fullsized":"2024-09-12T01:33:19.485555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486406+0000","last_change":"2024-09-12T01:33:16.594344+0000","last_active":"2024-09-12T01:33:19.486406+0000","last_peered":"2024-09-12T01:33:19.486406+0000","last_clean":"2024-09-12T01:33:19.486406+0000","last_became_active":"2024-09-12T01:33:16.593980+0000","last_became_peered":"2024-09-12T01:33:16.593980+0000","last_unstale":"2024-09-12T01:33:19.486406+0000","last_undegraded":"2024-09-12T01:33:19.486406+0000","last_fullsized":"2024-09-12T01:33:19.486406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:14.088579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218224+0000","last_change":"2024-09-12T01:33:12.170635+0000","last_active":"2024-09-12T01:33:18.218224+0000","last_peered":"2024-09-12T01:33:18.218224+0000","last_clean":"2024-09-12T01:33:18.218224+0000","last_became_active":"2024-09-12T01:33:12.169588+0000","last_became_peered":"2024-09-12T01:33:12.169588+0000","last_unstale":"2024-09-12T01:33:18.218224+0000","last_undegraded":"2024-09-12T01:33:18.218224+0000","last_fullsized":"2024-09-12T01:33:18.218224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:50.002525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486441+0000","last_change":"2024-09-12T01:33:11.151147+0000","last_active":"2024-09-12T01:33:19.486441+0000","last_peered":"2024-09-12T01:33:19.486441+0000","last_clean":"2024-09-12T01:33:19.486441+0000","last_became_active":"2024-09-12T01:33:11.150915+0000","last_became_peered":"2024-09-12T01:33:11.150915+0000","last_unstale":"2024-09-12T01:33:19.486441+0000","last_undegraded":"2024-09-12T01:33:19.486441+0000","last_fullsized":"2024-09-12T01:33:19.486441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:56.100575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218568+0000","last_change":"2024-09-12T01:33:16.202342+0000","last_active":"2024-09-12T01:33:18.218568+0000","last_peered":"2024-09-12T01:33:18.218568+0000","last_clean":"2024-09-12T01:33:18.218568+0000","last_became_active":"2024-09-12T01:33:16.200265+0000","last_became_peered":"2024-09-12T01:33:16.200265+0000","last_unstale":"2024-09-12T01:33:18.218568+0000","last_undegraded":"2024-09-12T01:33:18.218568+0000","last_fullsized":"2024-09-12T01:33:18.218568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:13.593524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413572+0000","last_change":"2024-09-12T01:33:12.171306+0000","last_active":"2024-09-12T01:33:19.413572+0000","last_peered":"2024-09-12T01:33:19.413572+0000","last_clean":"2024-09-12T01:33:19.413572+0000","last_became_active":"2024-09-12T01:33:12.171026+0000","last_became_peered":"2024-09-12T01:33:12.171026+0000","last_unstale":"2024-09-12T01:33:19.413572+0000","last_undegraded":"2024-09-12T01:33:19.413572+0000","last_fullsized":"2024-09-12T01:33:19.413572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:02.272439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487629+0000","last_change":"2024-09-12T01:33:11.153620+0000","last_active":"2024-09-12T01:33:19.487629+0000","last_peered":"2024-09-12T01:33:19.487629+0000","last_clean":"2024-09-12T01:33:19.487629+0000","last_became_active":"2024-09-12T01:33:11.153278+0000","last_became_peered":"2024-09-12T01:33:11.153278+0000","last_unstale":"2024-09-12T01:33:19.487629+0000","last_undegraded":"2024-09-12T01:33:19.487629+0000","last_fullsized":"2024-09-12T01:33:19.487629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217980+0000","last_change":"2024-09-12T01:33:16.590320+0000","last_active":"2024-09-12T01:33:18.217980+0000","last_peered":"2024-09-12T01:33:18.217980+0000","last_clean":"2024-09-12T01:33:18.217980+0000","last_became_active":"2024-09-12T01:33:16.589806+0000","last_became_peered":"2024-09-12T01:33:16.589806+0000","last_unstale":"2024-09-12T01:33:18.217980+0000","last_undegraded":"2024-09-12T01:33:18.217980+0000","last_fullsized":"2024-09-12T01:33:18.217980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:01.750852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217345+0000","last_change":"2024-09-12T01:33:12.177022+0000","last_active":"2024-09-12T01:33:18.217345+0000","last_peered":"2024-09-12T01:33:18.217345+0000","last_clean":"2024-09-12T01:33:18.217345+0000","last_became_active":"2024-09-12T01:33:12.176544+0000","last_became_peered":"2024-09-12T01:33:12.176544+0000","last_unstale":"2024-09-12T01:33:18.217345+0000","last_undegraded":"2024-09-12T01:33:18.217345+0000","last_fullsized":"2024-09-12T01:33:18.217345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:51.254891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486161+0000","last_change":"2024-09-12T01:33:11.144439+0000","last_active":"2024-09-12T01:33:19.486161+0000","last_peered":"2024-09-12T01:33:19.486161+0000","last_clean":"2024-09-12T01:33:19.486161+0000","last_became_active":"2024-09-12T01:33:11.144134+0000","last_became_peered":"2024-09-12T01:33:11.144134+0000","last_unstale":"2024-09-12T01:33:19.486161+0000","last_undegraded":"2024-09-12T01:33:19.486161+0000","last_fullsized":"2024-09-12T01:33:19.486161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216491+0000","last_change":"2024-09-12T01:33:16.588984+0000","last_active":"2024-09-12T01:33:18.216491+0000","last_peered":"2024-09-12T01:33:18.216491+0000","last_clean":"2024-09-12T01:33:18.216491+0000","last_became_active":"2024-09-12T01:33:16.588531+0000","last_became_peered":"2024-09-12T01:33:16.588531+0000","last_unstale":"2024-09-12T01:33:18.216491+0000","last_undegraded":"2024-09-12T01:33:18.216491+0000","last_fullsized":"2024-09-12T01:33:18.216491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:35.903448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486153+0000","last_change":"2024-09-12T01:33:12.162118+0000","last_active":"2024-09-12T01:33:19.486153+0000","last_peered":"2024-09-12T01:33:19.486153+0000","last_clean":"2024-09-12T01:33:19.486153+0000","last_became_active":"2024-09-12T01:33:12.161817+0000","last_became_peered":"2024-09-12T01:33:12.161817+0000","last_unstale":"2024-09-12T01:33:19.486153+0000","last_undegraded":"2024-09-12T01:33:19.486153+0000","last_fullsized":"2024-09-12T01:33:19.486153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:54.679349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487612+0000","last_change":"2024-09-12T01:33:11.150724+0000","last_active":"2024-09-12T01:33:19.487612+0000","last_peered":"2024-09-12T01:33:19.487612+0000","last_clean":"2024-09-12T01:33:19.487612+0000","last_became_active":"2024-09-12T01:33:11.150444+0000","last_became_peered":"2024-09-12T01:33:11.150444+0000","last_unstale":"2024-09-12T01:33:19.487612+0000","last_undegraded":"2024-09-12T01:33:19.487612+0000","last_fullsized":"2024-09-12T01:33:19.487612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:26.579494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413397+0000","last_change":"2024-09-12T01:33:16.589570+0000","last_active":"2024-09-12T01:33:19.413397+0000","last_peered":"2024-09-12T01:33:19.413397+0000","last_clean":"2024-09-12T01:33:19.413397+0000","last_became_active":"2024-09-12T01:33:16.589137+0000","last_became_peered":"2024-09-12T01:33:16.589137+0000","last_unstale":"2024-09-12T01:33:19.413397+0000","last_undegraded":"2024-09-12T01:33:19.413397+0000","last_fullsized":"2024-09-12T01:33:19.413397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:31:18.804222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839884+0000","last_change":"2024-09-12T01:33:12.174625+0000","last_active":"2024-09-12T01:33:18.839884+0000","last_peered":"2024-09-12T01:33:18.839884+0000","last_clean":"2024-09-12T01:33:18.839884+0000","last_became_active":"2024-09-12T01:33:12.174308+0000","last_became_peered":"2024-09-12T01:33:12.174308+0000","last_unstale":"2024-09-12T01:33:18.839884+0000","last_undegraded":"2024-09-12T01:33:18.839884+0000","last_fullsized":"2024-09-12T01:33:18.839884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:24.134321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486236+0000","last_change":"2024-09-12T01:33:11.142280+0000","last_active":"2024-09-12T01:33:19.486236+0000","last_peered":"2024-09-12T01:33:19.486236+0000","last_clean":"2024-09-12T01:33:19.486236+0000","last_became_active":"2024-09-12T01:33:11.141976+0000","last_became_peered":"2024-09-12T01:33:11.141976+0000","last_unstale":"2024-09-12T01:33:19.486236+0000","last_undegraded":"2024-09-12T01:33:19.486236+0000","last_fullsized":"2024-09-12T01:33:19.486236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:11.527363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485800+0000","last_change":"2024-09-12T01:33:16.200223+0000","last_active":"2024-09-12T01:33:19.485800+0000","last_peered":"2024-09-12T01:33:19.485800+0000","last_clean":"2024-09-12T01:33:19.485800+0000","last_became_active":"2024-09-12T01:33:16.199767+0000","last_became_peered":"2024-09-12T01:33:16.199767+0000","last_unstale":"2024-09-12T01:33:19.485800+0000","last_undegraded":"2024-09-12T01:33:19.485800+0000","last_fullsized":"2024-09-12T01:33:19.485800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:01.750852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218829+0000","last_change":"2024-09-12T01:33:12.171774+0000","last_active":"2024-09-12T01:33:18.218829+0000","last_peered":"2024-09-12T01:33:18.218829+0000","last_clean":"2024-09-12T01:33:18.218829+0000","last_became_active":"2024-09-12T01:33:12.171472+0000","last_became_peered":"2024-09-12T01:33:12.171472+0000","last_unstale":"2024-09-12T01:33:18.218829+0000","last_undegraded":"2024-09-12T01:33:18.218829+0000","last_fullsized":"2024-09-12T01:33:18.218829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:50.682111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839456+0000","last_change":"2024-09-12T01:33:11.146150+0000","last_active":"2024-09-12T01:33:18.839456+0000","last_peered":"2024-09-12T01:33:18.839456+0000","last_clean":"2024-09-12T01:33:18.839456+0000","last_became_active":"2024-09-12T01:33:11.145690+0000","last_became_peered":"2024-09-12T01:33:11.145690+0000","last_unstale":"2024-09-12T01:33:18.839456+0000","last_undegraded":"2024-09-12T01:33:18.839456+0000","last_fullsized":"2024-09-12T01:33:18.839456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:49.680053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486103+0000","last_change":"2024-09-12T01:33:16.200773+0000","last_active":"2024-09-12T01:33:19.486103+0000","last_peered":"2024-09-12T01:33:19.486103+0000","last_clean":"2024-09-12T01:33:19.486103+0000","last_became_active":"2024-09-12T01:33:16.199757+0000","last_became_peered":"2024-09-12T01:33:16.199757+0000","last_unstale":"2024-09-12T01:33:19.486103+0000","last_undegraded":"2024-09-12T01:33:19.486103+0000","last_fullsized":"2024-09-12T01:33:19.486103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:40.563121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485669+0000","last_change":"2024-09-12T01:33:12.162181+0000","last_active":"2024-09-12T01:33:19.485669+0000","last_peered":"2024-09-12T01:33:19.485669+0000","last_clean":"2024-09-12T01:33:19.485669+0000","last_became_active":"2024-09-12T01:33:12.161900+0000","last_became_peered":"2024-09-12T01:33:12.161900+0000","last_unstale":"2024-09-12T01:33:19.485669+0000","last_undegraded":"2024-09-12T01:33:19.485669+0000","last_fullsized":"2024-09-12T01:33:19.485669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:52.439908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554120+0000","last_change":"2024-09-12T01:33:11.147499+0000","last_active":"2024-09-12T01:33:18.554120+0000","last_peered":"2024-09-12T01:33:18.554120+0000","last_clean":"2024-09-12T01:33:18.554120+0000","last_became_active":"2024-09-12T01:33:11.147186+0000","last_became_peered":"2024-09-12T01:33:11.147186+0000","last_unstale":"2024-09-12T01:33:18.554120+0000","last_undegraded":"2024-09-12T01:33:18.554120+0000","last_fullsized":"2024-09-12T01:33:18.554120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:15.921424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486454+0000","last_change":"2024-09-12T01:33:16.201262+0000","last_active":"2024-09-12T01:33:19.486454+0000","last_peered":"2024-09-12T01:33:19.486454+0000","last_clean":"2024-09-12T01:33:19.486454+0000","last_became_active":"2024-09-12T01:33:16.200850+0000","last_became_peered":"2024-09-12T01:33:16.200850+0000","last_unstale":"2024-09-12T01:33:19.486454+0000","last_undegraded":"2024-09-12T01:33:19.486454+0000","last_fullsized":"2024-09-12T01:33:19.486454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:13.593524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.412948+0000","last_change":"2024-09-12T01:33:12.177601+0000","last_active":"2024-09-12T01:33:19.412948+0000","last_peered":"2024-09-12T01:33:19.412948+0000","last_clean":"2024-09-12T01:33:19.412948+0000","last_became_active":"2024-09-12T01:33:12.177310+0000","last_became_peered":"2024-09-12T01:33:12.177310+0000","last_unstale":"2024-09-12T01:33:19.412948+0000","last_undegraded":"2024-09-12T01:33:19.412948+0000","last_fullsized":"2024-09-12T01:33:19.412948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:59.541178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486689+0000","last_change":"2024-09-12T01:33:11.151233+0000","last_active":"2024-09-12T01:33:19.486689+0000","last_peered":"2024-09-12T01:33:19.486689+0000","last_clean":"2024-09-12T01:33:19.486689+0000","last_became_active":"2024-09-12T01:33:11.150960+0000","last_became_peered":"2024-09-12T01:33:11.150960+0000","last_unstale":"2024-09-12T01:33:19.486689+0000","last_undegraded":"2024-09-12T01:33:19.486689+0000","last_fullsized":"2024-09-12T01:33:19.486689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413558+0000","last_change":"2024-09-12T01:33:16.593340+0000","last_active":"2024-09-12T01:33:19.413558+0000","last_peered":"2024-09-12T01:33:19.413558+0000","last_clean":"2024-09-12T01:33:19.413558+0000","last_became_active":"2024-09-12T01:33:16.592942+0000","last_became_peered":"2024-09-12T01:33:16.592942+0000","last_unstale":"2024-09-12T01:33:19.413558+0000","last_undegraded":"2024-09-12T01:33:19.413558+0000","last_fullsized":"2024-09-12T01:33:19.413558+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:06.180609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216813+0000","last_change":"2024-09-12T01:33:12.175958+0000","last_active":"2024-09-12T01:33:18.216813+0000","last_peered":"2024-09-12T01:33:18.216813+0000","last_clean":"2024-09-12T01:33:18.216813+0000","last_became_active":"2024-09-12T01:33:12.174098+0000","last_became_peered":"2024-09-12T01:33:12.174098+0000","last_unstale":"2024-09-12T01:33:18.216813+0000","last_undegraded":"2024-09-12T01:33:18.216813+0000","last_fullsized":"2024-09-12T01:33:18.216813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839416+0000","last_change":"2024-09-12T01:33:11.148301+0000","last_active":"2024-09-12T01:33:18.839416+0000","last_peered":"2024-09-12T01:33:18.839416+0000","last_clean":"2024-09-12T01:33:18.839416+0000","last_became_active":"2024-09-12T01:33:11.146483+0000","last_became_peered":"2024-09-12T01:33:11.146483+0000","last_unstale":"2024-09-12T01:33:18.839416+0000","last_undegraded":"2024-09-12T01:33:18.839416+0000","last_fullsized":"2024-09-12T01:33:18.839416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:11.527363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486566+0000","last_change":"2024-09-12T01:33:01.096542+0000","last_active":"2024-09-12T01:33:19.486566+0000","last_peered":"2024-09-12T01:33:19.486566+0000","last_clean":"2024-09-12T01:33:19.486566+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:33:19.486566+0000","last_undegraded":"2024-09-12T01:33:19.486566+0000","last_fullsized":"2024-09-12T01:33:19.486566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_clean_scrub_stamp":"2024-09-12T01:32:58.512553+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:48:58.741813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486931+0000","last_change":"2024-09-12T01:33:16.591360+0000","last_active":"2024-09-12T01:33:19.486931+0000","last_peered":"2024-09-12T01:33:19.486931+0000","last_clean":"2024-09-12T01:33:19.486931+0000","last_became_active":"2024-09-12T01:33:16.590884+0000","last_became_peered":"2024-09-12T01:33:16.590884+0000","last_unstale":"2024-09-12T01:33:19.486931+0000","last_undegraded":"2024-09-12T01:33:19.486931+0000","last_fullsized":"2024-09-12T01:33:19.486931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:15.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487399+0000","last_change":"2024-09-12T01:33:12.178384+0000","last_active":"2024-09-12T01:33:19.487399+0000","last_peered":"2024-09-12T01:33:19.487399+0000","last_clean":"2024-09-12T01:33:19.487399+0000","last_became_active":"2024-09-12T01:33:12.178068+0000","last_became_peered":"2024-09-12T01:33:12.178068+0000","last_unstale":"2024-09-12T01:33:19.487399+0000","last_undegraded":"2024-09-12T01:33:19.487399+0000","last_fullsized":"2024-09-12T01:33:19.487399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:51.254891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554226+0000","last_change":"2024-09-12T01:33:11.149796+0000","last_active":"2024-09-12T01:33:18.554226+0000","last_peered":"2024-09-12T01:33:18.554226+0000","last_clean":"2024-09-12T01:33:18.554226+0000","last_became_active":"2024-09-12T01:33:11.149505+0000","last_became_peered":"2024-09-12T01:33:11.149505+0000","last_unstale":"2024-09-12T01:33:18.554226+0000","last_undegraded":"2024-09-12T01:33:18.554226+0000","last_fullsized":"2024-09-12T01:33:18.554226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:24.957500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486471+0000","last_change":"2024-09-12T01:33:16.589125+0000","last_active":"2024-09-12T01:33:19.486471+0000","last_peered":"2024-09-12T01:33:19.486471+0000","last_clean":"2024-09-12T01:33:19.486471+0000","last_became_active":"2024-09-12T01:33:16.588637+0000","last_became_peered":"2024-09-12T01:33:16.588637+0000","last_unstale":"2024-09-12T01:33:19.486471+0000","last_undegraded":"2024-09-12T01:33:19.486471+0000","last_fullsized":"2024-09-12T01:33:19.486471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:06.997893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839688+0000","last_change":"2024-09-12T01:33:12.176967+0000","last_active":"2024-09-12T01:33:18.839688+0000","last_peered":"2024-09-12T01:33:18.839688+0000","last_clean":"2024-09-12T01:33:18.839688+0000","last_became_active":"2024-09-12T01:33:12.176355+0000","last_became_peered":"2024-09-12T01:33:12.176355+0000","last_unstale":"2024-09-12T01:33:18.839688+0000","last_undegraded":"2024-09-12T01:33:18.839688+0000","last_fullsized":"2024-09-12T01:33:18.839688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:57.532127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485317+0000","last_change":"2024-09-12T01:33:11.149279+0000","last_active":"2024-09-12T01:33:19.485317+0000","last_peered":"2024-09-12T01:33:19.485317+0000","last_clean":"2024-09-12T01:33:19.485317+0000","last_became_active":"2024-09-12T01:33:11.148215+0000","last_became_peered":"2024-09-12T01:33:11.148215+0000","last_unstale":"2024-09-12T01:33:19.485317+0000","last_undegraded":"2024-09-12T01:33:19.485317+0000","last_fullsized":"2024-09-12T01:33:19.485317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:49.680053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413355+0000","last_change":"2024-09-12T01:33:16.200672+0000","last_active":"2024-09-12T01:33:19.413355+0000","last_peered":"2024-09-12T01:33:19.413355+0000","last_clean":"2024-09-12T01:33:19.413355+0000","last_became_active":"2024-09-12T01:33:16.200251+0000","last_became_peered":"2024-09-12T01:33:16.200251+0000","last_unstale":"2024-09-12T01:33:19.413355+0000","last_undegraded":"2024-09-12T01:33:19.413355+0000","last_fullsized":"2024-09-12T01:33:19.413355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:37.500171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487810+0000","last_change":"2024-09-12T01:33:12.179113+0000","last_active":"2024-09-12T01:33:19.487810+0000","last_peered":"2024-09-12T01:33:19.487810+0000","last_clean":"2024-09-12T01:33:19.487810+0000","last_became_active":"2024-09-12T01:33:12.178495+0000","last_became_peered":"2024-09-12T01:33:12.178495+0000","last_unstale":"2024-09-12T01:33:19.487810+0000","last_undegraded":"2024-09-12T01:33:19.487810+0000","last_fullsized":"2024-09-12T01:33:19.487810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:51:57.703184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554326+0000","last_change":"2024-09-12T01:33:11.144585+0000","last_active":"2024-09-12T01:33:18.554326+0000","last_peered":"2024-09-12T01:33:18.554326+0000","last_clean":"2024-09-12T01:33:18.554326+0000","last_became_active":"2024-09-12T01:33:11.144277+0000","last_became_peered":"2024-09-12T01:33:11.144277+0000","last_unstale":"2024-09-12T01:33:18.554326+0000","last_undegraded":"2024-09-12T01:33:18.554326+0000","last_fullsized":"2024-09-12T01:33:18.554326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:15.342457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839029+0000","last_change":"2024-09-12T01:33:16.591894+0000","last_active":"2024-09-12T01:33:18.839029+0000","last_peered":"2024-09-12T01:33:18.839029+0000","last_clean":"2024-09-12T01:33:18.839029+0000","last_became_active":"2024-09-12T01:33:16.591452+0000","last_became_peered":"2024-09-12T01:33:16.591452+0000","last_unstale":"2024-09-12T01:33:18.839029+0000","last_undegraded":"2024-09-12T01:33:18.839029+0000","last_fullsized":"2024-09-12T01:33:18.839029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:59.574294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487365+0000","last_change":"2024-09-12T01:33:12.176012+0000","last_active":"2024-09-12T01:33:19.487365+0000","last_peered":"2024-09-12T01:33:19.487365+0000","last_clean":"2024-09-12T01:33:19.487365+0000","last_became_active":"2024-09-12T01:33:12.175729+0000","last_became_peered":"2024-09-12T01:33:12.175729+0000","last_unstale":"2024-09-12T01:33:19.487365+0000","last_undegraded":"2024-09-12T01:33:19.487365+0000","last_fullsized":"2024-09-12T01:33:19.487365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:57.532127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.219006+0000","last_change":"2024-09-12T01:33:11.138837+0000","last_active":"2024-09-12T01:33:18.219006+0000","last_peered":"2024-09-12T01:33:18.219006+0000","last_clean":"2024-09-12T01:33:18.219006+0000","last_became_active":"2024-09-12T01:33:11.138324+0000","last_became_peered":"2024-09-12T01:33:11.138324+0000","last_unstale":"2024-09-12T01:33:18.219006+0000","last_undegraded":"2024-09-12T01:33:18.219006+0000","last_fullsized":"2024-09-12T01:33:18.219006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:48.706430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217732+0000","last_change":"2024-09-12T01:33:16.200428+0000","last_active":"2024-09-12T01:33:18.217732+0000","last_peered":"2024-09-12T01:33:18.217732+0000","last_clean":"2024-09-12T01:33:18.217732+0000","last_became_active":"2024-09-12T01:33:16.199943+0000","last_became_peered":"2024-09-12T01:33:16.199943+0000","last_unstale":"2024-09-12T01:33:18.217732+0000","last_undegraded":"2024-09-12T01:33:18.217732+0000","last_fullsized":"2024-09-12T01:33:18.217732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:15.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486512+0000","last_change":"2024-09-12T01:33:12.176546+0000","last_active":"2024-09-12T01:33:19.486512+0000","last_peered":"2024-09-12T01:33:19.486512+0000","last_clean":"2024-09-12T01:33:19.486512+0000","last_became_active":"2024-09-12T01:33:12.176254+0000","last_became_peered":"2024-09-12T01:33:12.176254+0000","last_unstale":"2024-09-12T01:33:19.486512+0000","last_undegraded":"2024-09-12T01:33:19.486512+0000","last_fullsized":"2024-09-12T01:33:19.486512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:24.363602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.553875+0000","last_change":"2024-09-12T01:33:11.143423+0000","last_active":"2024-09-12T01:33:18.553875+0000","last_peered":"2024-09-12T01:33:18.553875+0000","last_clean":"2024-09-12T01:33:18.553875+0000","last_became_active":"2024-09-12T01:33:11.143125+0000","last_became_peered":"2024-09-12T01:33:11.143125+0000","last_unstale":"2024-09-12T01:33:18.553875+0000","last_undegraded":"2024-09-12T01:33:18.553875+0000","last_fullsized":"2024-09-12T01:33:18.553875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:24.005795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413405+0000","last_change":"2024-09-12T01:33:16.190124+0000","last_active":"2024-09-12T01:33:19.413405+0000","last_peered":"2024-09-12T01:33:19.413405+0000","last_clean":"2024-09-12T01:33:19.413405+0000","last_became_active":"2024-09-12T01:33:16.189728+0000","last_became_peered":"2024-09-12T01:33:16.189728+0000","last_unstale":"2024-09-12T01:33:19.413405+0000","last_undegraded":"2024-09-12T01:33:19.413405+0000","last_fullsized":"2024-09-12T01:33:19.413405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:00.559557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486163+0000","last_change":"2024-09-12T01:33:12.173415+0000","last_active":"2024-09-12T01:33:19.486163+0000","last_peered":"2024-09-12T01:33:19.486163+0000","last_clean":"2024-09-12T01:33:19.486163+0000","last_became_active":"2024-09-12T01:33:12.173117+0000","last_became_peered":"2024-09-12T01:33:12.173117+0000","last_unstale":"2024-09-12T01:33:19.486163+0000","last_undegraded":"2024-09-12T01:33:19.486163+0000","last_fullsized":"2024-09-12T01:33:19.486163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:54.679349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486025+0000","last_change":"2024-09-12T01:33:11.140718+0000","last_active":"2024-09-12T01:33:19.486025+0000","last_peered":"2024-09-12T01:33:19.486025+0000","last_clean":"2024-09-12T01:33:19.486025+0000","last_became_active":"2024-09-12T01:33:11.140321+0000","last_became_peered":"2024-09-12T01:33:11.140321+0000","last_unstale":"2024-09-12T01:33:19.486025+0000","last_undegraded":"2024-09-12T01:33:19.486025+0000","last_fullsized":"2024-09-12T01:33:19.486025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:22.885518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217295+0000","last_change":"2024-09-12T01:33:16.594875+0000","last_active":"2024-09-12T01:33:18.217295+0000","last_peered":"2024-09-12T01:33:18.217295+0000","last_clean":"2024-09-12T01:33:18.217295+0000","last_became_active":"2024-09-12T01:33:16.592733+0000","last_became_peered":"2024-09-12T01:33:16.592733+0000","last_unstale":"2024-09-12T01:33:18.217295+0000","last_undegraded":"2024-09-12T01:33:18.217295+0000","last_fullsized":"2024-09-12T01:33:18.217295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:39.080379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486853+0000","last_change":"2024-09-12T01:33:12.175983+0000","last_active":"2024-09-12T01:33:19.486853+0000","last_peered":"2024-09-12T01:33:19.486853+0000","last_clean":"2024-09-12T01:33:19.486853+0000","last_became_active":"2024-09-12T01:33:12.175638+0000","last_became_peered":"2024-09-12T01:33:12.175638+0000","last_unstale":"2024-09-12T01:33:19.486853+0000","last_undegraded":"2024-09-12T01:33:19.486853+0000","last_fullsized":"2024-09-12T01:33:19.486853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:48.292206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839432+0000","last_change":"2024-09-12T01:33:11.148832+0000","last_active":"2024-09-12T01:33:18.839432+0000","last_peered":"2024-09-12T01:33:18.839432+0000","last_clean":"2024-09-12T01:33:18.839432+0000","last_became_active":"2024-09-12T01:33:11.148519+0000","last_became_peered":"2024-09-12T01:33:11.148519+0000","last_unstale":"2024-09-12T01:33:18.839432+0000","last_undegraded":"2024-09-12T01:33:18.839432+0000","last_fullsized":"2024-09-12T01:33:18.839432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217452+0000","last_change":"2024-09-12T01:33:16.588958+0000","last_active":"2024-09-12T01:33:18.217452+0000","last_peered":"2024-09-12T01:33:18.217452+0000","last_clean":"2024-09-12T01:33:18.217452+0000","last_became_active":"2024-09-12T01:33:16.588510+0000","last_became_peered":"2024-09-12T01:33:16.588510+0000","last_unstale":"2024-09-12T01:33:18.217452+0000","last_undegraded":"2024-09-12T01:33:18.217452+0000","last_fullsized":"2024-09-12T01:33:18.217452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:00.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413221+0000","last_change":"2024-09-12T01:33:12.184393+0000","last_active":"2024-09-12T01:33:19.413221+0000","last_peered":"2024-09-12T01:33:19.413221+0000","last_clean":"2024-09-12T01:33:19.413221+0000","last_became_active":"2024-09-12T01:33:12.184103+0000","last_became_peered":"2024-09-12T01:33:12.184103+0000","last_unstale":"2024-09-12T01:33:19.413221+0000","last_undegraded":"2024-09-12T01:33:19.413221+0000","last_fullsized":"2024-09-12T01:33:19.413221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:36.026056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.412683+0000","last_change":"2024-09-12T01:33:11.134020+0000","last_active":"2024-09-12T01:33:19.412683+0000","last_peered":"2024-09-12T01:33:19.412683+0000","last_clean":"2024-09-12T01:33:19.412683+0000","last_became_active":"2024-09-12T01:33:11.133717+0000","last_became_peered":"2024-09-12T01:33:11.133717+0000","last_unstale":"2024-09-12T01:33:19.412683+0000","last_undegraded":"2024-09-12T01:33:19.412683+0000","last_fullsized":"2024-09-12T01:33:19.412683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:20.901290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839735+0000","last_change":"2024-09-12T01:33:16.593051+0000","last_active":"2024-09-12T01:33:18.839735+0000","last_peered":"2024-09-12T01:33:18.839735+0000","last_clean":"2024-09-12T01:33:18.839735+0000","last_became_active":"2024-09-12T01:33:16.592513+0000","last_became_peered":"2024-09-12T01:33:16.592513+0000","last_unstale":"2024-09-12T01:33:18.839735+0000","last_undegraded":"2024-09-12T01:33:18.839735+0000","last_fullsized":"2024-09-12T01:33:18.839735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:35.903448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487266+0000","last_change":"2024-09-12T01:33:12.178596+0000","last_active":"2024-09-12T01:33:19.487266+0000","last_peered":"2024-09-12T01:33:19.487266+0000","last_clean":"2024-09-12T01:33:19.487266+0000","last_became_active":"2024-09-12T01:33:12.176316+0000","last_became_peered":"2024-09-12T01:33:12.176316+0000","last_unstale":"2024-09-12T01:33:19.487266+0000","last_undegraded":"2024-09-12T01:33:19.487266+0000","last_fullsized":"2024-09-12T01:33:19.487266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:24.134321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:21.709458+0000","last_change":"2024-09-12T01:33:11.147275+0000","last_active":"2024-09-12T01:33:21.709458+0000","last_peered":"2024-09-12T01:33:21.709458+0000","last_clean":"2024-09-12T01:33:21.709458+0000","last_became_active":"2024-09-12T01:33:11.146962+0000","last_became_peered":"2024-09-12T01:33:11.146962+0000","last_unstale":"2024-09-12T01:33:21.709458+0000","last_undegraded":"2024-09-12T01:33:21.709458+0000","last_fullsized":"2024-09-12T01:33:21.709458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:11.527363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485898+0000","last_change":"2024-09-12T01:33:16.202736+0000","last_active":"2024-09-12T01:33:19.485898+0000","last_peered":"2024-09-12T01:33:19.485898+0000","last_clean":"2024-09-12T01:33:19.485898+0000","last_became_active":"2024-09-12T01:33:16.202282+0000","last_became_peered":"2024-09-12T01:33:16.202282+0000","last_unstale":"2024-09-12T01:33:19.485898+0000","last_undegraded":"2024-09-12T01:33:19.485898+0000","last_fullsized":"2024-09-12T01:33:19.485898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:28.321407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218580+0000","last_change":"2024-09-12T01:33:12.173488+0000","last_active":"2024-09-12T01:33:18.218580+0000","last_peered":"2024-09-12T01:33:18.218580+0000","last_clean":"2024-09-12T01:33:18.218580+0000","last_became_active":"2024-09-12T01:33:12.173187+0000","last_became_peered":"2024-09-12T01:33:12.173187+0000","last_unstale":"2024-09-12T01:33:18.218580+0000","last_undegraded":"2024-09-12T01:33:18.218580+0000","last_fullsized":"2024-09-12T01:33:18.218580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:27.581552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485330+0000","last_change":"2024-09-12T01:33:11.149565+0000","last_active":"2024-09-12T01:33:19.485330+0000","last_peered":"2024-09-12T01:33:19.485330+0000","last_clean":"2024-09-12T01:33:19.485330+0000","last_became_active":"2024-09-12T01:33:11.149034+0000","last_became_peered":"2024-09-12T01:33:11.149034+0000","last_unstale":"2024-09-12T01:33:19.485330+0000","last_undegraded":"2024-09-12T01:33:19.485330+0000","last_fullsized":"2024-09-12T01:33:19.485330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218212+0000","last_change":"2024-09-12T01:33:16.202414+0000","last_active":"2024-09-12T01:33:18.218212+0000","last_peered":"2024-09-12T01:33:18.218212+0000","last_clean":"2024-09-12T01:33:18.218212+0000","last_became_active":"2024-09-12T01:33:16.200143+0000","last_became_peered":"2024-09-12T01:33:16.200143+0000","last_unstale":"2024-09-12T01:33:18.218212+0000","last_undegraded":"2024-09-12T01:33:18.218212+0000","last_fullsized":"2024-09-12T01:33:18.218212+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:14.088579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413595+0000","last_change":"2024-09-12T01:33:12.174901+0000","last_active":"2024-09-12T01:33:19.413595+0000","last_peered":"2024-09-12T01:33:19.413595+0000","last_clean":"2024-09-12T01:33:19.413595+0000","last_became_active":"2024-09-12T01:33:12.174638+0000","last_became_peered":"2024-09-12T01:33:12.174638+0000","last_unstale":"2024-09-12T01:33:19.413595+0000","last_undegraded":"2024-09-12T01:33:19.413595+0000","last_fullsized":"2024-09-12T01:33:19.413595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:54.169299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839752+0000","last_change":"2024-09-12T01:33:11.147387+0000","last_active":"2024-09-12T01:33:18.839752+0000","last_peered":"2024-09-12T01:33:18.839752+0000","last_clean":"2024-09-12T01:33:18.839752+0000","last_became_active":"2024-09-12T01:33:11.147110+0000","last_became_peered":"2024-09-12T01:33:11.147110+0000","last_unstale":"2024-09-12T01:33:18.839752+0000","last_undegraded":"2024-09-12T01:33:18.839752+0000","last_fullsized":"2024-09-12T01:33:18.839752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216657+0000","last_change":"2024-09-12T01:33:16.191800+0000","last_active":"2024-09-12T01:33:18.216657+0000","last_peered":"2024-09-12T01:33:18.216657+0000","last_clean":"2024-09-12T01:33:18.216657+0000","last_became_active":"2024-09-12T01:33:16.190448+0000","last_became_peered":"2024-09-12T01:33:16.190448+0000","last_unstale":"2024-09-12T01:33:18.216657+0000","last_undegraded":"2024-09-12T01:33:18.216657+0000","last_fullsized":"2024-09-12T01:33:18.216657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:47.099767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839819+0000","last_change":"2024-09-12T01:33:12.178000+0000","last_active":"2024-09-12T01:33:18.839819+0000","last_peered":"2024-09-12T01:33:18.839819+0000","last_clean":"2024-09-12T01:33:18.839819+0000","last_became_active":"2024-09-12T01:33:12.177703+0000","last_became_peered":"2024-09-12T01:33:12.177703+0000","last_unstale":"2024-09-12T01:33:18.839819+0000","last_undegraded":"2024-09-12T01:33:18.839819+0000","last_fullsized":"2024-09-12T01:33:18.839819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:56.959747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485907+0000","last_change":"2024-09-12T01:33:11.149688+0000","last_active":"2024-09-12T01:33:19.485907+0000","last_peered":"2024-09-12T01:33:19.485907+0000","last_clean":"2024-09-12T01:33:19.485907+0000","last_became_active":"2024-09-12T01:33:11.149363+0000","last_became_peered":"2024-09-12T01:33:11.149363+0000","last_unstale":"2024-09-12T01:33:19.485907+0000","last_undegraded":"2024-09-12T01:33:19.485907+0000","last_fullsized":"2024-09-12T01:33:19.485907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216625+0000","last_change":"2024-09-12T01:33:16.592951+0000","last_active":"2024-09-12T01:33:18.216625+0000","last_peered":"2024-09-12T01:33:18.216625+0000","last_clean":"2024-09-12T01:33:18.216625+0000","last_became_active":"2024-09-12T01:33:16.592558+0000","last_became_peered":"2024-09-12T01:33:16.592558+0000","last_unstale":"2024-09-12T01:33:18.216625+0000","last_undegraded":"2024-09-12T01:33:18.216625+0000","last_fullsized":"2024-09-12T01:33:18.216625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:29.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413385+0000","last_change":"2024-09-12T01:33:16.594318+0000","last_active":"2024-09-12T01:33:19.413385+0000","last_peered":"2024-09-12T01:33:19.413385+0000","last_clean":"2024-09-12T01:33:19.413385+0000","last_became_active":"2024-09-12T01:33:16.593834+0000","last_became_peered":"2024-09-12T01:33:16.593834+0000","last_unstale":"2024-09-12T01:33:19.413385+0000","last_undegraded":"2024-09-12T01:33:19.413385+0000","last_fullsized":"2024-09-12T01:33:19.413385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:11.126504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485462+0000","last_change":"2024-09-12T01:33:12.171282+0000","last_active":"2024-09-12T01:33:19.485462+0000","last_peered":"2024-09-12T01:33:19.485462+0000","last_clean":"2024-09-12T01:33:19.485462+0000","last_became_active":"2024-09-12T01:33:12.170988+0000","last_became_peered":"2024-09-12T01:33:12.170988+0000","last_unstale":"2024-09-12T01:33:19.485462+0000","last_undegraded":"2024-09-12T01:33:19.485462+0000","last_fullsized":"2024-09-12T01:33:19.485462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:25.007983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839177+0000","last_change":"2024-09-12T01:33:11.146497+0000","last_active":"2024-09-12T01:33:18.839177+0000","last_peered":"2024-09-12T01:33:18.839177+0000","last_clean":"2024-09-12T01:33:18.839177+0000","last_became_active":"2024-09-12T01:33:11.146184+0000","last_became_peered":"2024-09-12T01:33:11.146184+0000","last_unstale":"2024-09-12T01:33:18.839177+0000","last_undegraded":"2024-09-12T01:33:18.839177+0000","last_fullsized":"2024-09-12T01:33:18.839177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487513+0000","last_change":"2024-09-12T01:33:16.202584+0000","last_active":"2024-09-12T01:33:19.487513+0000","last_peered":"2024-09-12T01:33:19.487513+0000","last_clean":"2024-09-12T01:33:19.487513+0000","last_became_active":"2024-09-12T01:33:16.202137+0000","last_became_peered":"2024-09-12T01:33:16.202137+0000","last_unstale":"2024-09-12T01:33:19.487513+0000","last_undegraded":"2024-09-12T01:33:19.487513+0000","last_fullsized":"2024-09-12T01:33:19.487513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:39.155565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216613+0000","last_change":"2024-09-12T01:33:12.169183+0000","last_active":"2024-09-12T01:33:18.216613+0000","last_peered":"2024-09-12T01:33:18.216613+0000","last_clean":"2024-09-12T01:33:18.216613+0000","last_became_active":"2024-09-12T01:33:12.168803+0000","last_became_peered":"2024-09-12T01:33:12.168803+0000","last_unstale":"2024-09-12T01:33:18.216613+0000","last_undegraded":"2024-09-12T01:33:18.216613+0000","last_fullsized":"2024-09-12T01:33:18.216613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:25.007983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216934+0000","last_change":"2024-09-12T01:33:11.144264+0000","last_active":"2024-09-12T01:33:18.216934+0000","last_peered":"2024-09-12T01:33:18.216934+0000","last_clean":"2024-09-12T01:33:18.216934+0000","last_became_active":"2024-09-12T01:33:11.143993+0000","last_became_peered":"2024-09-12T01:33:11.143993+0000","last_unstale":"2024-09-12T01:33:18.216934+0000","last_undegraded":"2024-09-12T01:33:18.216934+0000","last_fullsized":"2024-09-12T01:33:18.216934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:02:33.335627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485661+0000","last_change":"2024-09-12T01:33:11.143410+0000","last_active":"2024-09-12T01:33:19.485661+0000","last_peered":"2024-09-12T01:33:19.485661+0000","last_clean":"2024-09-12T01:33:19.485661+0000","last_became_active":"2024-09-12T01:33:11.143121+0000","last_became_peered":"2024-09-12T01:33:11.143121+0000","last_unstale":"2024-09-12T01:33:19.485661+0000","last_undegraded":"2024-09-12T01:33:19.485661+0000","last_fullsized":"2024-09-12T01:33:19.485661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413493+0000","last_change":"2024-09-12T01:33:12.176891+0000","last_active":"2024-09-12T01:33:19.413493+0000","last_peered":"2024-09-12T01:33:19.413493+0000","last_clean":"2024-09-12T01:33:19.413493+0000","last_became_active":"2024-09-12T01:33:12.176573+0000","last_became_peered":"2024-09-12T01:33:12.176573+0000","last_unstale":"2024-09-12T01:33:19.413493+0000","last_undegraded":"2024-09-12T01:33:19.413493+0000","last_fullsized":"2024-09-12T01:33:19.413493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:47.157354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554126+0000","last_change":"2024-09-12T01:33:16.195123+0000","last_active":"2024-09-12T01:33:18.554126+0000","last_peered":"2024-09-12T01:33:18.554126+0000","last_clean":"2024-09-12T01:33:18.554126+0000","last_became_active":"2024-09-12T01:33:16.194690+0000","last_became_peered":"2024-09-12T01:33:16.194690+0000","last_unstale":"2024-09-12T01:33:18.554126+0000","last_undegraded":"2024-09-12T01:33:18.554126+0000","last_fullsized":"2024-09-12T01:33:18.554126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:14.296077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217027+0000","last_change":"2024-09-12T01:33:11.145695+0000","last_active":"2024-09-12T01:33:18.217027+0000","last_peered":"2024-09-12T01:33:18.217027+0000","last_clean":"2024-09-12T01:33:18.217027+0000","last_became_active":"2024-09-12T01:33:11.145391+0000","last_became_peered":"2024-09-12T01:33:11.145391+0000","last_unstale":"2024-09-12T01:33:18.217027+0000","last_undegraded":"2024-09-12T01:33:18.217027+0000","last_fullsized":"2024-09-12T01:33:18.217027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:47.695596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218383+0000","last_change":"2024-09-12T01:33:12.164156+0000","last_active":"2024-09-12T01:33:18.218383+0000","last_peered":"2024-09-12T01:33:18.218383+0000","last_clean":"2024-09-12T01:33:18.218383+0000","last_became_active":"2024-09-12T01:33:12.160798+0000","last_became_peered":"2024-09-12T01:33:12.160798+0000","last_unstale":"2024-09-12T01:33:18.218383+0000","last_undegraded":"2024-09-12T01:33:18.218383+0000","last_fullsized":"2024-09-12T01:33:18.218383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:56.394003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217262+0000","last_change":"2024-09-12T01:33:16.198377+0000","last_active":"2024-09-12T01:33:18.217262+0000","last_peered":"2024-09-12T01:33:18.217262+0000","last_clean":"2024-09-12T01:33:18.217262+0000","last_became_active":"2024-09-12T01:33:16.197913+0000","last_became_peered":"2024-09-12T01:33:16.197913+0000","last_unstale":"2024-09-12T01:33:18.217262+0000","last_undegraded":"2024-09-12T01:33:18.217262+0000","last_fullsized":"2024-09-12T01:33:18.217262+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:56:05.124457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486041+0000","last_change":"2024-09-12T01:33:12.176137+0000","last_active":"2024-09-12T01:33:19.486041+0000","last_peered":"2024-09-12T01:33:19.486041+0000","last_clean":"2024-09-12T01:33:19.486041+0000","last_became_active":"2024-09-12T01:33:12.175840+0000","last_became_peered":"2024-09-12T01:33:12.175840+0000","last_unstale":"2024-09-12T01:33:19.486041+0000","last_undegraded":"2024-09-12T01:33:19.486041+0000","last_fullsized":"2024-09-12T01:33:19.486041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:38.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554012+0000","last_change":"2024-09-12T01:33:12.168134+0000","last_active":"2024-09-12T01:33:18.554012+0000","last_peered":"2024-09-12T01:33:18.554012+0000","last_clean":"2024-09-12T01:33:18.554012+0000","last_became_active":"2024-09-12T01:33:12.167844+0000","last_became_peered":"2024-09-12T01:33:12.167844+0000","last_unstale":"2024-09-12T01:33:18.554012+0000","last_undegraded":"2024-09-12T01:33:18.554012+0000","last_fullsized":"2024-09-12T01:33:18.554012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:06.175990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839339+0000","last_change":"2024-09-12T01:33:12.183112+0000","last_active":"2024-09-12T01:33:18.839339+0000","last_peered":"2024-09-12T01:33:18.839339+0000","last_clean":"2024-09-12T01:33:18.839339+0000","last_became_active":"2024-09-12T01:33:12.182858+0000","last_became_peered":"2024-09-12T01:33:12.182858+0000","last_unstale":"2024-09-12T01:33:18.839339+0000","last_undegraded":"2024-09-12T01:33:18.839339+0000","last_fullsized":"2024-09-12T01:33:18.839339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:11.508027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413461+0000","last_change":"2024-09-12T01:33:12.181683+0000","last_active":"2024-09-12T01:33:19.413461+0000","last_peered":"2024-09-12T01:33:19.413461+0000","last_clean":"2024-09-12T01:33:19.413461+0000","last_became_active":"2024-09-12T01:33:12.181383+0000","last_became_peered":"2024-09-12T01:33:12.181383+0000","last_unstale":"2024-09-12T01:33:19.413461+0000","last_undegraded":"2024-09-12T01:33:19.413461+0000","last_fullsized":"2024-09-12T01:33:19.413461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:00.865640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217578+0000","last_change":"2024-09-12T01:33:12.178479+0000","last_active":"2024-09-12T01:33:18.217578+0000","last_peered":"2024-09-12T01:33:18.217578+0000","last_clean":"2024-09-12T01:33:18.217578+0000","last_became_active":"2024-09-12T01:33:12.178198+0000","last_became_peered":"2024-09-12T01:33:12.178198+0000","last_unstale":"2024-09-12T01:33:18.217578+0000","last_undegraded":"2024-09-12T01:33:18.217578+0000","last_fullsized":"2024-09-12T01:33:18.217578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:53.398489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485933+0000","last_change":"2024-09-12T01:33:12.168261+0000","last_active":"2024-09-12T01:33:19.485933+0000","last_peered":"2024-09-12T01:33:19.485933+0000","last_clean":"2024-09-12T01:33:19.485933+0000","last_became_active":"2024-09-12T01:33:12.167980+0000","last_became_peered":"2024-09-12T01:33:12.167980+0000","last_unstale":"2024-09-12T01:33:19.485933+0000","last_undegraded":"2024-09-12T01:33:19.485933+0000","last_fullsized":"2024-09-12T01:33:19.485933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:02.393918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487377+0000","last_change":"2024-09-12T01:33:12.186333+0000","last_active":"2024-09-12T01:33:19.487377+0000","last_peered":"2024-09-12T01:33:19.487377+0000","last_clean":"2024-09-12T01:33:19.487377+0000","last_became_active":"2024-09-12T01:33:12.186047+0000","last_became_peered":"2024-09-12T01:33:12.186047+0000","last_unstale":"2024-09-12T01:33:19.487377+0000","last_undegraded":"2024-09-12T01:33:19.487377+0000","last_fullsized":"2024-09-12T01:33:19.487377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:53.398489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217203+0000","last_change":"2024-09-12T01:33:12.177911+0000","last_active":"2024-09-12T01:33:18.217203+0000","last_peered":"2024-09-12T01:33:18.217203+0000","last_clean":"2024-09-12T01:33:18.217203+0000","last_became_active":"2024-09-12T01:33:12.177625+0000","last_became_peered":"2024-09-12T01:33:12.177625+0000","last_unstale":"2024-09-12T01:33:18.217203+0000","last_undegraded":"2024-09-12T01:33:18.217203+0000","last_fullsized":"2024-09-12T01:33:18.217203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:02.393918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839173+0000","last_change":"2024-09-12T01:33:12.183935+0000","last_active":"2024-09-12T01:33:18.839173+0000","last_peered":"2024-09-12T01:33:18.839173+0000","last_clean":"2024-09-12T01:33:18.839173+0000","last_became_active":"2024-09-12T01:33:12.183620+0000","last_became_peered":"2024-09-12T01:33:12.183620+0000","last_unstale":"2024-09-12T01:33:18.839173+0000","last_undegraded":"2024-09-12T01:33:18.839173+0000","last_fullsized":"2024-09-12T01:33:18.839173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:02.950225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216744+0000","last_change":"2024-09-12T01:33:12.179130+0000","last_active":"2024-09-12T01:33:18.216744+0000","last_peered":"2024-09-12T01:33:18.216744+0000","last_clean":"2024-09-12T01:33:18.216744+0000","last_became_active":"2024-09-12T01:33:12.177141+0000","last_became_peered":"2024-09-12T01:33:12.177141+0000","last_unstale":"2024-09-12T01:33:18.216744+0000","last_undegraded":"2024-09-12T01:33:18.216744+0000","last_fullsized":"2024-09-12T01:33:18.216744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:24.134321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839306+0000","last_change":"2024-09-12T01:33:12.175899+0000","last_active":"2024-09-12T01:33:18.839306+0000","last_peered":"2024-09-12T01:33:18.839306+0000","last_clean":"2024-09-12T01:33:18.839306+0000","last_became_active":"2024-09-12T01:33:12.175594+0000","last_became_peered":"2024-09-12T01:33:12.175594+0000","last_unstale":"2024-09-12T01:33:18.839306+0000","last_undegraded":"2024-09-12T01:33:18.839306+0000","last_fullsized":"2024-09-12T01:33:18.839306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:51.254891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413194+0000","last_change":"2024-09-12T01:33:12.185737+0000","last_active":"2024-09-12T01:33:19.413194+0000","last_peered":"2024-09-12T01:33:19.413194+0000","last_clean":"2024-09-12T01:33:19.413194+0000","last_became_active":"2024-09-12T01:33:12.185489+0000","last_became_peered":"2024-09-12T01:33:12.185489+0000","last_unstale":"2024-09-12T01:33:19.413194+0000","last_undegraded":"2024-09-12T01:33:19.413194+0000","last_fullsized":"2024-09-12T01:33:19.413194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:05.301039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486993+0000","last_change":"2024-09-12T01:33:12.183324+0000","last_active":"2024-09-12T01:33:19.486993+0000","last_peered":"2024-09-12T01:33:19.486993+0000","last_clean":"2024-09-12T01:33:19.486993+0000","last_became_active":"2024-09-12T01:33:12.182857+0000","last_became_peered":"2024-09-12T01:33:12.182857+0000","last_unstale":"2024-09-12T01:33:19.486993+0000","last_undegraded":"2024-09-12T01:33:19.486993+0000","last_fullsized":"2024-09-12T01:33:19.486993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:52.439908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487522+0000","last_change":"2024-09-12T01:33:12.177539+0000","last_active":"2024-09-12T01:33:19.487522+0000","last_peered":"2024-09-12T01:33:19.487522+0000","last_clean":"2024-09-12T01:33:19.487522+0000","last_became_active":"2024-09-12T01:33:12.173980+0000","last_became_peered":"2024-09-12T01:33:12.173980+0000","last_unstale":"2024-09-12T01:33:19.487522+0000","last_undegraded":"2024-09-12T01:33:19.487522+0000","last_fullsized":"2024-09-12T01:33:19.487522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:11.508027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554334+0000","last_change":"2024-09-12T01:33:12.179198+0000","last_active":"2024-09-12T01:33:18.554334+0000","last_peered":"2024-09-12T01:33:18.554334+0000","last_clean":"2024-09-12T01:33:18.554334+0000","last_became_active":"2024-09-12T01:33:12.178859+0000","last_became_peered":"2024-09-12T01:33:12.178859+0000","last_unstale":"2024-09-12T01:33:18.554334+0000","last_undegraded":"2024-09-12T01:33:18.554334+0000","last_fullsized":"2024-09-12T01:33:18.554334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:46.228922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839997+0000","last_change":"2024-09-12T01:33:12.179001+0000","last_active":"2024-09-12T01:33:18.839997+0000","last_peered":"2024-09-12T01:33:18.839997+0000","last_clean":"2024-09-12T01:33:18.839997+0000","last_became_active":"2024-09-12T01:33:12.178298+0000","last_became_peered":"2024-09-12T01:33:12.178298+0000","last_unstale":"2024-09-12T01:33:18.839997+0000","last_undegraded":"2024-09-12T01:33:18.839997+0000","last_fullsized":"2024-09-12T01:33:18.839997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:35.107896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839003+0000","last_change":"2024-09-12T01:33:12.169863+0000","last_active":"2024-09-12T01:33:18.839003+0000","last_peered":"2024-09-12T01:33:18.839003+0000","last_clean":"2024-09-12T01:33:18.839003+0000","last_became_active":"2024-09-12T01:33:12.169441+0000","last_became_peered":"2024-09-12T01:33:12.169441+0000","last_unstale":"2024-09-12T01:33:18.839003+0000","last_undegraded":"2024-09-12T01:33:18.839003+0000","last_fullsized":"2024-09-12T01:33:18.839003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485524+0000","last_change":"2024-09-12T01:33:12.160854+0000","last_active":"2024-09-12T01:33:19.485524+0000","last_peered":"2024-09-12T01:33:19.485524+0000","last_clean":"2024-09-12T01:33:19.485524+0000","last_became_active":"2024-09-12T01:33:12.160512+0000","last_became_peered":"2024-09-12T01:33:12.160512+0000","last_unstale":"2024-09-12T01:33:19.485524+0000","last_undegraded":"2024-09-12T01:33:19.485524+0000","last_fullsized":"2024-09-12T01:33:19.485524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:38.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413165+0000","last_change":"2024-09-12T01:33:12.172031+0000","last_active":"2024-09-12T01:33:19.413165+0000","last_peered":"2024-09-12T01:33:19.413165+0000","last_clean":"2024-09-12T01:33:19.413165+0000","last_became_active":"2024-09-12T01:33:12.171730+0000","last_became_peered":"2024-09-12T01:33:12.171730+0000","last_unstale":"2024-09-12T01:33:19.413165+0000","last_undegraded":"2024-09-12T01:33:19.413165+0000","last_fullsized":"2024-09-12T01:33:19.413165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:10.248408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216814+0000","last_change":"2024-09-12T01:33:12.173718+0000","last_active":"2024-09-12T01:33:18.216814+0000","last_peered":"2024-09-12T01:33:18.216814+0000","last_clean":"2024-09-12T01:33:18.216814+0000","last_became_active":"2024-09-12T01:33:12.171379+0000","last_became_peered":"2024-09-12T01:33:12.171379+0000","last_unstale":"2024-09-12T01:33:18.216814+0000","last_undegraded":"2024-09-12T01:33:18.216814+0000","last_fullsized":"2024-09-12T01:33:18.216814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:52.439908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218338+0000","last_change":"2024-09-12T01:33:12.174895+0000","last_active":"2024-09-12T01:33:18.218338+0000","last_peered":"2024-09-12T01:33:18.218338+0000","last_clean":"2024-09-12T01:33:18.218338+0000","last_became_active":"2024-09-12T01:33:12.174591+0000","last_became_peered":"2024-09-12T01:33:12.174591+0000","last_unstale":"2024-09-12T01:33:18.218338+0000","last_undegraded":"2024-09-12T01:33:18.218338+0000","last_fullsized":"2024-09-12T01:33:18.218338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:57.102634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486963+0000","last_change":"2024-09-12T01:33:12.181588+0000","last_active":"2024-09-12T01:33:19.486963+0000","last_peered":"2024-09-12T01:33:19.486963+0000","last_clean":"2024-09-12T01:33:19.486963+0000","last_became_active":"2024-09-12T01:33:12.181293+0000","last_became_peered":"2024-09-12T01:33:12.181293+0000","last_unstale":"2024-09-12T01:33:19.486963+0000","last_undegraded":"2024-09-12T01:33:19.486963+0000","last_fullsized":"2024-09-12T01:33:19.486963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:02.393918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487308+0000","last_change":"2024-09-12T01:33:12.175792+0000","last_active":"2024-09-12T01:33:19.487308+0000","last_peered":"2024-09-12T01:33:19.487308+0000","last_clean":"2024-09-12T01:33:19.487308+0000","last_became_active":"2024-09-12T01:33:12.175332+0000","last_became_peered":"2024-09-12T01:33:12.175332+0000","last_unstale":"2024-09-12T01:33:19.487308+0000","last_undegraded":"2024-09-12T01:33:19.487308+0000","last_fullsized":"2024-09-12T01:33:19.487308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487796+0000","last_change":"2024-09-12T01:33:12.178831+0000","last_active":"2024-09-12T01:33:19.487796+0000","last_peered":"2024-09-12T01:33:19.487796+0000","last_clean":"2024-09-12T01:33:19.487796+0000","last_became_active":"2024-09-12T01:33:12.178116+0000","last_became_peered":"2024-09-12T01:33:12.178116+0000","last_unstale":"2024-09-12T01:33:19.487796+0000","last_undegraded":"2024-09-12T01:33:19.487796+0000","last_fullsized":"2024-09-12T01:33:19.487796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:35.107896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216514+0000","last_change":"2024-09-12T01:33:12.172225+0000","last_active":"2024-09-12T01:33:18.216514+0000","last_peered":"2024-09-12T01:33:18.216514+0000","last_clean":"2024-09-12T01:33:18.216514+0000","last_became_active":"2024-09-12T01:33:12.171933+0000","last_became_peered":"2024-09-12T01:33:12.171933+0000","last_unstale":"2024-09-12T01:33:18.216514+0000","last_undegraded":"2024-09-12T01:33:18.216514+0000","last_fullsized":"2024-09-12T01:33:18.216514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:20.867318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486602+0000","last_change":"2024-09-12T01:33:12.173395+0000","last_active":"2024-09-12T01:33:19.486602+0000","last_peered":"2024-09-12T01:33:19.486602+0000","last_clean":"2024-09-12T01:33:19.486602+0000","last_became_active":"2024-09-12T01:33:12.173090+0000","last_became_peered":"2024-09-12T01:33:12.173090+0000","last_unstale":"2024-09-12T01:33:19.486602+0000","last_undegraded":"2024-09-12T01:33:19.486602+0000","last_fullsized":"2024-09-12T01:33:19.486602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:46.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839213+0000","last_change":"2024-09-12T01:33:12.169604+0000","last_active":"2024-09-12T01:33:18.839213+0000","last_peered":"2024-09-12T01:33:18.839213+0000","last_clean":"2024-09-12T01:33:18.839213+0000","last_became_active":"2024-09-12T01:33:12.169258+0000","last_became_peered":"2024-09-12T01:33:12.169258+0000","last_unstale":"2024-09-12T01:33:18.839213+0000","last_undegraded":"2024-09-12T01:33:18.839213+0000","last_fullsized":"2024-09-12T01:33:18.839213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:46.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486571+0000","last_change":"2024-09-12T01:33:12.171033+0000","last_active":"2024-09-12T01:33:19.486571+0000","last_peered":"2024-09-12T01:33:19.486571+0000","last_clean":"2024-09-12T01:33:19.486571+0000","last_became_active":"2024-09-12T01:33:12.170708+0000","last_became_peered":"2024-09-12T01:33:12.170708+0000","last_unstale":"2024-09-12T01:33:19.486571+0000","last_undegraded":"2024-09-12T01:33:19.486571+0000","last_fullsized":"2024-09-12T01:33:19.486571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:56.959747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839539+0000","last_change":"2024-09-12T01:33:12.182780+0000","last_active":"2024-09-12T01:33:18.839539+0000","last_peered":"2024-09-12T01:33:18.839539+0000","last_clean":"2024-09-12T01:33:18.839539+0000","last_became_active":"2024-09-12T01:33:12.182454+0000","last_became_peered":"2024-09-12T01:33:12.182454+0000","last_unstale":"2024-09-12T01:33:18.839539+0000","last_undegraded":"2024-09-12T01:33:18.839539+0000","last_fullsized":"2024-09-12T01:33:18.839539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:53.398489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413295+0000","last_change":"2024-09-12T01:33:12.180723+0000","last_active":"2024-09-12T01:33:19.413295+0000","last_peered":"2024-09-12T01:33:19.413295+0000","last_clean":"2024-09-12T01:33:19.413295+0000","last_became_active":"2024-09-12T01:33:12.180414+0000","last_became_peered":"2024-09-12T01:33:12.180414+0000","last_unstale":"2024-09-12T01:33:19.413295+0000","last_undegraded":"2024-09-12T01:33:19.413295+0000","last_fullsized":"2024-09-12T01:33:19.413295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:58.901902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486128+0000","last_change":"2024-09-12T01:33:12.165745+0000","last_active":"2024-09-12T01:33:19.486128+0000","last_peered":"2024-09-12T01:33:19.486128+0000","last_clean":"2024-09-12T01:33:19.486128+0000","last_became_active":"2024-09-12T01:33:12.165475+0000","last_became_peered":"2024-09-12T01:33:12.165475+0000","last_unstale":"2024-09-12T01:33:19.486128+0000","last_undegraded":"2024-09-12T01:33:19.486128+0000","last_fullsized":"2024-09-12T01:33:19.486128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487685+0000","last_change":"2024-09-12T01:33:12.179105+0000","last_active":"2024-09-12T01:33:19.487685+0000","last_peered":"2024-09-12T01:33:19.487685+0000","last_clean":"2024-09-12T01:33:19.487685+0000","last_became_active":"2024-09-12T01:33:12.173779+0000","last_became_peered":"2024-09-12T01:33:12.173779+0000","last_unstale":"2024-09-12T01:33:19.487685+0000","last_undegraded":"2024-09-12T01:33:19.487685+0000","last_fullsized":"2024-09-12T01:33:19.487685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:02.950225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":700890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":113784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":110030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509452,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":110066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":282624,"data_stored":113784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":110100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":270336,"data_stored":109940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":876544,"data_stored":700946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:33:36.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:33:36.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.788+0000 7fbe36827640 1 Processor -- start 2024-09-12T01:33:36.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.788+0000 7fbe36827640 1 -- start start 2024-09-12T01:33:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.789+0000 7fbe36827640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe300d1550 0x7fbe301691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.789+0000 7fbe36827640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe300d3480 con 0x7fbe300ceee0 2024-09-12T01:33:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.789+0000 7fbe36827640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe300d3760 con 0x7fbe3015e560 2024-09-12T01:33:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.789+0000 7fbe36827640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbe300d3a40 con 0x7fbe300d1550 2024-09-12T01:33:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.789+0000 7fbe34dc4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe300d1550 0x7fbe301691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.789+0000 7fbe2f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbe300ceee0 0x7fbe300d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46168/0 (socket says 172.21.15.16:46168) 2024-09-12T01:33:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.789+0000 7fbe2f7fe640 1 -- 172.21.15.16:0/2053524016 learned_addr learned my addr 172.21.15.16:0/2053524016 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.789+0000 7fbe34dc4640 1 -- 172.21.15.16:0/2053524016 >> v1:172.21.15.33:6789/0 conn(0x7fbe3015e560 legacy=0x7fbe300cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.789+0000 7fbe34dc4640 1 -- 172.21.15.16:0/2053524016 >> v1:172.21.15.16:6789/0 conn(0x7fbe300ceee0 legacy=0x7fbe300d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.789+0000 7fbe34dc4640 1 -- 172.21.15.16:0/2053524016 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe300d3d20 con 0x7fbe300d1550 2024-09-12T01:33:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.790+0000 7fbe34dc4640 1 --2- 172.21.15.16:0/2053524016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe300d1550 0x7fbe301691c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009d90 tx=0x7fbe2802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d92644d15a61de5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.790+0000 7fbe2effd640 1 -- 172.21.15.16:0/2053524016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28007740 con 0x7fbe300d1550 2024-09-12T01:33:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.790+0000 7fbe2effd640 1 -- 172.21.15.16:0/2053524016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe280078e0 con 0x7fbe300d1550 2024-09-12T01:33:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.790+0000 7fbe2effd640 1 -- 172.21.15.16:0/2053524016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28007cb0 con 0x7fbe300d1550 2024-09-12T01:33:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.790+0000 7fbe36827640 1 -- 172.21.15.16:0/2053524016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe300d1550 msgr2=0x7fbe301691c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.791+0000 7fbe36827640 1 --2- 172.21.15.16:0/2053524016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe300d1550 0x7fbe301691c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009d90 tx=0x7fbe2802a6f0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.791+0000 7fbe36827640 1 -- 172.21.15.16:0/2053524016 shutdown_connections 2024-09-12T01:33:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.791+0000 7fbe36827640 1 --2- 172.21.15.16:0/2053524016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe300d1550 0x7fbe301691c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.791+0000 7fbe36827640 1 -- 172.21.15.16:0/2053524016 >> 172.21.15.16:0/2053524016 conn(0x7fbe30073850 msgr2=0x7fbe3006cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.791+0000 7fbe36827640 1 -- 172.21.15.16:0/2053524016 shutdown_connections 2024-09-12T01:33:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.791+0000 7fbe36827640 1 -- 172.21.15.16:0/2053524016 wait complete. 2024-09-12T01:33:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.792+0000 7fbe36827640 1 Processor -- start 2024-09-12T01:33:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.792+0000 7fbe36827640 1 -- start start 2024-09-12T01:33:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe36827640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300ceee0 0x7fbe30144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe36827640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbe300d1550 0x7fbe30144f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe36827640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3015e560 0x7fbe301454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe36827640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3015eac0 con 0x7fbe3015e560 2024-09-12T01:33:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe36827640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3015e940 con 0x7fbe300ceee0 2024-09-12T01:33:36.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe36827640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fbe300b87a0 con 0x7fbe300d1550 2024-09-12T01:33:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe34dc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3015e560 0x7fbe301454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe34dc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3015e560 0x7fbe301454a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60398/0 (socket says 172.21.15.16:60398) 2024-09-12T01:33:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe34dc4640 1 -- 172.21.15.16:0/2601218255 learned_addr learned my addr 172.21.15.16:0/2601218255 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe2ffff640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300ceee0 0x7fbe30144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe2f7fe640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbe300d1550 0x7fbe30144f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe34dc4640 1 -- 172.21.15.16:0/2601218255 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbe300d1550 msgr2=0x7fbe30144f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe34dc4640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbe300d1550 0x7fbe30144f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe34dc4640 1 -- 172.21.15.16:0/2601218255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300ceee0 msgr2=0x7fbe30144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe34dc4640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300ceee0 0x7fbe30144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe34dc4640 1 -- 172.21.15.16:0/2601218255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe300b4a30 con 0x7fbe3015e560 2024-09-12T01:33:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe34dc4640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3015e560 0x7fbe301454a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009eb0 tx=0x7fbe28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.793+0000 7fbe2cff9640 1 -- 172.21.15.16:0/2601218255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28040070 con 0x7fbe3015e560 2024-09-12T01:33:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.794+0000 7fbe2cff9640 1 -- 172.21.15.16:0/2601218255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe28031c90 con 0x7fbe3015e560 2024-09-12T01:33:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.794+0000 7fbe36827640 1 -- 172.21.15.16:0/2601218255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe300b4d10 con 0x7fbe3015e560 2024-09-12T01:33:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.794+0000 7fbe2cff9640 1 -- 172.21.15.16:0/2601218255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28045680 con 0x7fbe3015e560 2024-09-12T01:33:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.794+0000 7fbe36827640 1 -- 172.21.15.16:0/2601218255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe300b52d0 con 0x7fbe3015e560 2024-09-12T01:33:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.795+0000 7fbe36827640 1 -- 172.21.15.16:0/2601218255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe300d0e40 con 0x7fbe3015e560 2024-09-12T01:33:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.795+0000 7fbe2cff9640 1 -- 172.21.15.16:0/2601218255 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe28039040 con 0x7fbe3015e560 2024-09-12T01:33:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.796+0000 7fbe2cff9640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbe1c06be30 0x7fbe1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.796+0000 7fbe2cff9640 1 -- 172.21.15.16:0/2601218255 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbe280acaa0 con 0x7fbe3015e560 2024-09-12T01:33:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.796+0000 7fbe2ffff640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbe1c06be30 0x7fbe1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.796+0000 7fbe2ffff640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbe1c06be30 0x7fbe1c06e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbe200016e0 tx=0x7fbe20009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.800+0000 7fbe2cff9640 1 -- 172.21.15.16:0/2601218255 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbe2807ab80 con 0x7fbe3015e560 2024-09-12T01:33:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.976+0000 7fbe36827640 1 -- 172.21.15.16:0/2601218255 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe3015aff0 con 0x7fbe1c06be30 2024-09-12T01:33:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.984+0000 7fbe2cff9640 1 -- 172.21.15.16:0/2601218255 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fbe3015aff0 con 0x7fbe1c06be30 2024-09-12T01:33:36.987 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:33:36.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.988+0000 7fbe36827640 1 -- 172.21.15.16:0/2601218255 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbe1c06be30 msgr2=0x7fbe1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.988+0000 7fbe36827640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbe1c06be30 0x7fbe1c06e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbe200016e0 tx=0x7fbe20009040 comp rx=0 tx=0).stop 2024-09-12T01:33:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.988+0000 7fbe36827640 1 -- 172.21.15.16:0/2601218255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3015e560 msgr2=0x7fbe301454a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.988+0000 7fbe36827640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3015e560 0x7fbe301454a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009eb0 tx=0x7fbe28004a40 comp rx=0 tx=0).stop 2024-09-12T01:33:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.988+0000 7fbe36827640 1 -- 172.21.15.16:0/2601218255 shutdown_connections 2024-09-12T01:33:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.988+0000 7fbe36827640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbe1c06be30 0x7fbe1c06e2f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.988+0000 7fbe36827640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3015e560 0x7fbe301454a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.988+0000 7fbe36827640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbe300d1550 0x7fbe30144f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.988+0000 7fbe36827640 1 --2- 172.21.15.16:0/2601218255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300ceee0 0x7fbe30144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.988+0000 7fbe36827640 1 -- 172.21.15.16:0/2601218255 >> 172.21.15.16:0/2601218255 conn(0x7fbe30073850 msgr2=0x7fbe300b8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.988+0000 7fbe36827640 1 -- 172.21.15.16:0/2601218255 shutdown_connections 2024-09-12T01:33:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:36.988+0000 7fbe36827640 1 -- 172.21.15.16:0/2601218255 wait complete. 2024-09-12T01:33:36.998 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-12T01:33:36.475954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236204,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738516,"statfs":{"total":773094113280,"available":772852240384,"internally_reserved":0,"allocated":3416064,"data_stored":2069540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003204"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413650+0000","last_change":"2024-09-12T01:33:11.148986+0000","last_active":"2024-09-12T01:33:19.413650+0000","last_peered":"2024-09-12T01:33:19.413650+0000","last_clean":"2024-09-12T01:33:19.413650+0000","last_became_active":"2024-09-12T01:33:11.146323+0000","last_became_peered":"2024-09-12T01:33:11.146323+0000","last_unstale":"2024-09-12T01:33:19.413650+0000","last_undegraded":"2024-09-12T01:33:19.413650+0000","last_fullsized":"2024-09-12T01:33:19.413650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:54.118009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217034+0000","last_change":"2024-09-12T01:33:12.183084+0000","last_active":"2024-09-12T01:33:18.217034+0000","last_peered":"2024-09-12T01:33:18.217034+0000","last_clean":"2024-09-12T01:33:18.217034+0000","last_became_active":"2024-09-12T01:33:12.182286+0000","last_became_peered":"2024-09-12T01:33:12.182286+0000","last_unstale":"2024-09-12T01:33:18.217034+0000","last_undegraded":"2024-09-12T01:33:18.217034+0000","last_fullsized":"2024-09-12T01:33:18.217034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:46.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216567+0000","last_change":"2024-09-12T01:33:16.593060+0000","last_active":"2024-09-12T01:33:18.216567+0000","last_peered":"2024-09-12T01:33:18.216567+0000","last_clean":"2024-09-12T01:33:18.216567+0000","last_became_active":"2024-09-12T01:33:16.592584+0000","last_became_peered":"2024-09-12T01:33:16.592584+0000","last_unstale":"2024-09-12T01:33:18.216567+0000","last_undegraded":"2024-09-12T01:33:18.216567+0000","last_fullsized":"2024-09-12T01:33:18.216567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:57.525309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217133+0000","last_change":"2024-09-12T01:33:11.142022+0000","last_active":"2024-09-12T01:33:18.217133+0000","last_peered":"2024-09-12T01:33:18.217133+0000","last_clean":"2024-09-12T01:33:18.217133+0000","last_became_active":"2024-09-12T01:33:11.141709+0000","last_became_peered":"2024-09-12T01:33:11.141709+0000","last_unstale":"2024-09-12T01:33:18.217133+0000","last_undegraded":"2024-09-12T01:33:18.217133+0000","last_fullsized":"2024-09-12T01:33:18.217133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:04.872785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413419+0000","last_change":"2024-09-12T01:33:12.176163+0000","last_active":"2024-09-12T01:33:19.413419+0000","last_peered":"2024-09-12T01:33:19.413419+0000","last_clean":"2024-09-12T01:33:19.413419+0000","last_became_active":"2024-09-12T01:33:12.175795+0000","last_became_peered":"2024-09-12T01:33:12.175795+0000","last_unstale":"2024-09-12T01:33:19.413419+0000","last_undegraded":"2024-09-12T01:33:19.413419+0000","last_fullsized":"2024-09-12T01:33:19.413419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:29.557033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217872+0000","last_change":"2024-09-12T01:33:16.202792+0000","last_active":"2024-09-12T01:33:18.217872+0000","last_peered":"2024-09-12T01:33:18.217872+0000","last_clean":"2024-09-12T01:33:18.217872+0000","last_became_active":"2024-09-12T01:33:16.202352+0000","last_became_peered":"2024-09-12T01:33:16.202352+0000","last_unstale":"2024-09-12T01:33:18.217872+0000","last_undegraded":"2024-09-12T01:33:18.217872+0000","last_fullsized":"2024-09-12T01:33:18.217872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:39.155565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487520+0000","last_change":"2024-09-12T01:33:11.149265+0000","last_active":"2024-09-12T01:33:19.487520+0000","last_peered":"2024-09-12T01:33:19.487520+0000","last_clean":"2024-09-12T01:33:19.487520+0000","last_became_active":"2024-09-12T01:33:11.148975+0000","last_became_peered":"2024-09-12T01:33:11.148975+0000","last_unstale":"2024-09-12T01:33:19.487520+0000","last_undegraded":"2024-09-12T01:33:19.487520+0000","last_fullsized":"2024-09-12T01:33:19.487520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:49.680053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839630+0000","last_change":"2024-09-12T01:33:12.175597+0000","last_active":"2024-09-12T01:33:18.839630+0000","last_peered":"2024-09-12T01:33:18.839630+0000","last_clean":"2024-09-12T01:33:18.839630+0000","last_became_active":"2024-09-12T01:33:12.175271+0000","last_became_peered":"2024-09-12T01:33:12.175271+0000","last_unstale":"2024-09-12T01:33:18.839630+0000","last_undegraded":"2024-09-12T01:33:18.839630+0000","last_fullsized":"2024-09-12T01:33:18.839630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:24.363602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218697+0000","last_change":"2024-09-12T01:33:16.200812+0000","last_active":"2024-09-12T01:33:18.218697+0000","last_peered":"2024-09-12T01:33:18.218697+0000","last_clean":"2024-09-12T01:33:18.218697+0000","last_became_active":"2024-09-12T01:33:16.200029+0000","last_became_peered":"2024-09-12T01:33:16.200029+0000","last_unstale":"2024-09-12T01:33:18.218697+0000","last_undegraded":"2024-09-12T01:33:18.218697+0000","last_fullsized":"2024-09-12T01:33:18.218697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:19.443781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485752+0000","last_change":"2024-09-12T01:33:11.153223+0000","last_active":"2024-09-12T01:33:19.485752+0000","last_peered":"2024-09-12T01:33:19.485752+0000","last_clean":"2024-09-12T01:33:19.485752+0000","last_became_active":"2024-09-12T01:33:11.152938+0000","last_became_peered":"2024-09-12T01:33:11.152938+0000","last_unstale":"2024-09-12T01:33:19.485752+0000","last_undegraded":"2024-09-12T01:33:19.485752+0000","last_fullsized":"2024-09-12T01:33:19.485752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839743+0000","last_change":"2024-09-12T01:33:12.178087+0000","last_active":"2024-09-12T01:33:18.839743+0000","last_peered":"2024-09-12T01:33:18.839743+0000","last_clean":"2024-09-12T01:33:18.839743+0000","last_became_active":"2024-09-12T01:33:12.176556+0000","last_became_peered":"2024-09-12T01:33:12.176556+0000","last_unstale":"2024-09-12T01:33:18.839743+0000","last_undegraded":"2024-09-12T01:33:18.839743+0000","last_fullsized":"2024-09-12T01:33:18.839743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:48.292206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218149+0000","last_change":"2024-09-12T01:33:16.193585+0000","last_active":"2024-09-12T01:33:18.218149+0000","last_peered":"2024-09-12T01:33:18.218149+0000","last_clean":"2024-09-12T01:33:18.218149+0000","last_became_active":"2024-09-12T01:33:16.193050+0000","last_became_peered":"2024-09-12T01:33:16.193050+0000","last_unstale":"2024-09-12T01:33:18.218149+0000","last_undegraded":"2024-09-12T01:33:18.218149+0000","last_fullsized":"2024-09-12T01:33:18.218149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:40.563121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486654+0000","last_change":"2024-09-12T01:33:12.178078+0000","last_active":"2024-09-12T01:33:19.486654+0000","last_peered":"2024-09-12T01:33:19.486654+0000","last_clean":"2024-09-12T01:33:19.486654+0000","last_became_active":"2024-09-12T01:33:12.177806+0000","last_became_peered":"2024-09-12T01:33:12.177806+0000","last_unstale":"2024-09-12T01:33:19.486654+0000","last_undegraded":"2024-09-12T01:33:19.486654+0000","last_fullsized":"2024-09-12T01:33:19.486654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:25.007983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485717+0000","last_change":"2024-09-12T01:33:11.149750+0000","last_active":"2024-09-12T01:33:19.485717+0000","last_peered":"2024-09-12T01:33:19.485717+0000","last_clean":"2024-09-12T01:33:19.485717+0000","last_became_active":"2024-09-12T01:33:11.149442+0000","last_became_peered":"2024-09-12T01:33:11.149442+0000","last_unstale":"2024-09-12T01:33:19.485717+0000","last_undegraded":"2024-09-12T01:33:19.485717+0000","last_fullsized":"2024-09-12T01:33:19.485717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218830+0000","last_change":"2024-09-12T01:33:16.202498+0000","last_active":"2024-09-12T01:33:18.218830+0000","last_peered":"2024-09-12T01:33:18.218830+0000","last_clean":"2024-09-12T01:33:18.218830+0000","last_became_active":"2024-09-12T01:33:16.200800+0000","last_became_peered":"2024-09-12T01:33:16.200800+0000","last_unstale":"2024-09-12T01:33:18.218830+0000","last_undegraded":"2024-09-12T01:33:18.218830+0000","last_fullsized":"2024-09-12T01:33:18.218830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:53.648825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216927+0000","last_change":"2024-09-12T01:33:12.169842+0000","last_active":"2024-09-12T01:33:18.216927+0000","last_peered":"2024-09-12T01:33:18.216927+0000","last_clean":"2024-09-12T01:33:18.216927+0000","last_became_active":"2024-09-12T01:33:12.169524+0000","last_became_peered":"2024-09-12T01:33:12.169524+0000","last_unstale":"2024-09-12T01:33:18.216927+0000","last_undegraded":"2024-09-12T01:33:18.216927+0000","last_fullsized":"2024-09-12T01:33:18.216927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:38.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485299+0000","last_change":"2024-09-12T01:33:11.147221+0000","last_active":"2024-09-12T01:33:19.485299+0000","last_peered":"2024-09-12T01:33:19.485299+0000","last_clean":"2024-09-12T01:33:19.485299+0000","last_became_active":"2024-09-12T01:33:11.146928+0000","last_became_peered":"2024-09-12T01:33:11.146928+0000","last_unstale":"2024-09-12T01:33:19.485299+0000","last_undegraded":"2024-09-12T01:33:19.485299+0000","last_fullsized":"2024-09-12T01:33:19.485299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:22.885518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218288+0000","last_change":"2024-09-12T01:33:16.196589+0000","last_active":"2024-09-12T01:33:18.218288+0000","last_peered":"2024-09-12T01:33:18.218288+0000","last_clean":"2024-09-12T01:33:18.218288+0000","last_became_active":"2024-09-12T01:33:16.196092+0000","last_became_peered":"2024-09-12T01:33:16.196092+0000","last_unstale":"2024-09-12T01:33:18.218288+0000","last_undegraded":"2024-09-12T01:33:18.218288+0000","last_fullsized":"2024-09-12T01:33:18.218288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:28.321407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216715+0000","last_change":"2024-09-12T01:33:12.165178+0000","last_active":"2024-09-12T01:33:18.216715+0000","last_peered":"2024-09-12T01:33:18.216715+0000","last_clean":"2024-09-12T01:33:18.216715+0000","last_became_active":"2024-09-12T01:33:12.162823+0000","last_became_peered":"2024-09-12T01:33:12.162823+0000","last_unstale":"2024-09-12T01:33:18.216715+0000","last_undegraded":"2024-09-12T01:33:18.216715+0000","last_fullsized":"2024-09-12T01:33:18.216715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:54.679349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485555+0000","last_change":"2024-09-12T01:33:11.145358+0000","last_active":"2024-09-12T01:33:19.485555+0000","last_peered":"2024-09-12T01:33:19.485555+0000","last_clean":"2024-09-12T01:33:19.485555+0000","last_became_active":"2024-09-12T01:33:11.145035+0000","last_became_peered":"2024-09-12T01:33:11.145035+0000","last_unstale":"2024-09-12T01:33:19.485555+0000","last_undegraded":"2024-09-12T01:33:19.485555+0000","last_fullsized":"2024-09-12T01:33:19.485555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486406+0000","last_change":"2024-09-12T01:33:16.594344+0000","last_active":"2024-09-12T01:33:19.486406+0000","last_peered":"2024-09-12T01:33:19.486406+0000","last_clean":"2024-09-12T01:33:19.486406+0000","last_became_active":"2024-09-12T01:33:16.593980+0000","last_became_peered":"2024-09-12T01:33:16.593980+0000","last_unstale":"2024-09-12T01:33:19.486406+0000","last_undegraded":"2024-09-12T01:33:19.486406+0000","last_fullsized":"2024-09-12T01:33:19.486406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:14.088579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218224+0000","last_change":"2024-09-12T01:33:12.170635+0000","last_active":"2024-09-12T01:33:18.218224+0000","last_peered":"2024-09-12T01:33:18.218224+0000","last_clean":"2024-09-12T01:33:18.218224+0000","last_became_active":"2024-09-12T01:33:12.169588+0000","last_became_peered":"2024-09-12T01:33:12.169588+0000","last_unstale":"2024-09-12T01:33:18.218224+0000","last_undegraded":"2024-09-12T01:33:18.218224+0000","last_fullsized":"2024-09-12T01:33:18.218224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:50.002525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486441+0000","last_change":"2024-09-12T01:33:11.151147+0000","last_active":"2024-09-12T01:33:19.486441+0000","last_peered":"2024-09-12T01:33:19.486441+0000","last_clean":"2024-09-12T01:33:19.486441+0000","last_became_active":"2024-09-12T01:33:11.150915+0000","last_became_peered":"2024-09-12T01:33:11.150915+0000","last_unstale":"2024-09-12T01:33:19.486441+0000","last_undegraded":"2024-09-12T01:33:19.486441+0000","last_fullsized":"2024-09-12T01:33:19.486441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:56.100575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218568+0000","last_change":"2024-09-12T01:33:16.202342+0000","last_active":"2024-09-12T01:33:18.218568+0000","last_peered":"2024-09-12T01:33:18.218568+0000","last_clean":"2024-09-12T01:33:18.218568+0000","last_became_active":"2024-09-12T01:33:16.200265+0000","last_became_peered":"2024-09-12T01:33:16.200265+0000","last_unstale":"2024-09-12T01:33:18.218568+0000","last_undegraded":"2024-09-12T01:33:18.218568+0000","last_fullsized":"2024-09-12T01:33:18.218568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:13.593524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413572+0000","last_change":"2024-09-12T01:33:12.171306+0000","last_active":"2024-09-12T01:33:19.413572+0000","last_peered":"2024-09-12T01:33:19.413572+0000","last_clean":"2024-09-12T01:33:19.413572+0000","last_became_active":"2024-09-12T01:33:12.171026+0000","last_became_peered":"2024-09-12T01:33:12.171026+0000","last_unstale":"2024-09-12T01:33:19.413572+0000","last_undegraded":"2024-09-12T01:33:19.413572+0000","last_fullsized":"2024-09-12T01:33:19.413572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:02.272439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487629+0000","last_change":"2024-09-12T01:33:11.153620+0000","last_active":"2024-09-12T01:33:19.487629+0000","last_peered":"2024-09-12T01:33:19.487629+0000","last_clean":"2024-09-12T01:33:19.487629+0000","last_became_active":"2024-09-12T01:33:11.153278+0000","last_became_peered":"2024-09-12T01:33:11.153278+0000","last_unstale":"2024-09-12T01:33:19.487629+0000","last_undegraded":"2024-09-12T01:33:19.487629+0000","last_fullsized":"2024-09-12T01:33:19.487629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217980+0000","last_change":"2024-09-12T01:33:16.590320+0000","last_active":"2024-09-12T01:33:18.217980+0000","last_peered":"2024-09-12T01:33:18.217980+0000","last_clean":"2024-09-12T01:33:18.217980+0000","last_became_active":"2024-09-12T01:33:16.589806+0000","last_became_peered":"2024-09-12T01:33:16.589806+0000","last_unstale":"2024-09-12T01:33:18.217980+0000","last_undegraded":"2024-09-12T01:33:18.217980+0000","last_fullsized":"2024-09-12T01:33:18.217980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:01.750852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217345+0000","last_change":"2024-09-12T01:33:12.177022+0000","last_active":"2024-09-12T01:33:18.217345+0000","last_peered":"2024-09-12T01:33:18.217345+0000","last_clean":"2024-09-12T01:33:18.217345+0000","last_became_active":"2024-09-12T01:33:12.176544+0000","last_became_peered":"2024-09-12T01:33:12.176544+0000","last_unstale":"2024-09-12T01:33:18.217345+0000","last_undegraded":"2024-09-12T01:33:18.217345+0000","last_fullsized":"2024-09-12T01:33:18.217345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:51.254891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486161+0000","last_change":"2024-09-12T01:33:11.144439+0000","last_active":"2024-09-12T01:33:19.486161+0000","last_peered":"2024-09-12T01:33:19.486161+0000","last_clean":"2024-09-12T01:33:19.486161+0000","last_became_active":"2024-09-12T01:33:11.144134+0000","last_became_peered":"2024-09-12T01:33:11.144134+0000","last_unstale":"2024-09-12T01:33:19.486161+0000","last_undegraded":"2024-09-12T01:33:19.486161+0000","last_fullsized":"2024-09-12T01:33:19.486161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216491+0000","last_change":"2024-09-12T01:33:16.588984+0000","last_active":"2024-09-12T01:33:18.216491+0000","last_peered":"2024-09-12T01:33:18.216491+0000","last_clean":"2024-09-12T01:33:18.216491+0000","last_became_active":"2024-09-12T01:33:16.588531+0000","last_became_peered":"2024-09-12T01:33:16.588531+0000","last_unstale":"2024-09-12T01:33:18.216491+0000","last_undegraded":"2024-09-12T01:33:18.216491+0000","last_fullsized":"2024-09-12T01:33:18.216491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:35.903448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486153+0000","last_change":"2024-09-12T01:33:12.162118+0000","last_active":"2024-09-12T01:33:19.486153+0000","last_peered":"2024-09-12T01:33:19.486153+0000","last_clean":"2024-09-12T01:33:19.486153+0000","last_became_active":"2024-09-12T01:33:12.161817+0000","last_became_peered":"2024-09-12T01:33:12.161817+0000","last_unstale":"2024-09-12T01:33:19.486153+0000","last_undegraded":"2024-09-12T01:33:19.486153+0000","last_fullsized":"2024-09-12T01:33:19.486153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:54.679349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487612+0000","last_change":"2024-09-12T01:33:11.150724+0000","last_active":"2024-09-12T01:33:19.487612+0000","last_peered":"2024-09-12T01:33:19.487612+0000","last_clean":"2024-09-12T01:33:19.487612+0000","last_became_active":"2024-09-12T01:33:11.150444+0000","last_became_peered":"2024-09-12T01:33:11.150444+0000","last_unstale":"2024-09-12T01:33:19.487612+0000","last_undegraded":"2024-09-12T01:33:19.487612+0000","last_fullsized":"2024-09-12T01:33:19.487612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:26.579494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413397+0000","last_change":"2024-09-12T01:33:16.589570+0000","last_active":"2024-09-12T01:33:19.413397+0000","last_peered":"2024-09-12T01:33:19.413397+0000","last_clean":"2024-09-12T01:33:19.413397+0000","last_became_active":"2024-09-12T01:33:16.589137+0000","last_became_peered":"2024-09-12T01:33:16.589137+0000","last_unstale":"2024-09-12T01:33:19.413397+0000","last_undegraded":"2024-09-12T01:33:19.413397+0000","last_fullsized":"2024-09-12T01:33:19.413397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:31:18.804222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839884+0000","last_change":"2024-09-12T01:33:12.174625+0000","last_active":"2024-09-12T01:33:18.839884+0000","last_peered":"2024-09-12T01:33:18.839884+0000","last_clean":"2024-09-12T01:33:18.839884+0000","last_became_active":"2024-09-12T01:33:12.174308+0000","last_became_peered":"2024-09-12T01:33:12.174308+0000","last_unstale":"2024-09-12T01:33:18.839884+0000","last_undegraded":"2024-09-12T01:33:18.839884+0000","last_fullsized":"2024-09-12T01:33:18.839884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:24.134321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486236+0000","last_change":"2024-09-12T01:33:11.142280+0000","last_active":"2024-09-12T01:33:19.486236+0000","last_peered":"2024-09-12T01:33:19.486236+0000","last_clean":"2024-09-12T01:33:19.486236+0000","last_became_active":"2024-09-12T01:33:11.141976+0000","last_became_peered":"2024-09-12T01:33:11.141976+0000","last_unstale":"2024-09-12T01:33:19.486236+0000","last_undegraded":"2024-09-12T01:33:19.486236+0000","last_fullsized":"2024-09-12T01:33:19.486236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:11.527363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485800+0000","last_change":"2024-09-12T01:33:16.200223+0000","last_active":"2024-09-12T01:33:19.485800+0000","last_peered":"2024-09-12T01:33:19.485800+0000","last_clean":"2024-09-12T01:33:19.485800+0000","last_became_active":"2024-09-12T01:33:16.199767+0000","last_became_peered":"2024-09-12T01:33:16.199767+0000","last_unstale":"2024-09-12T01:33:19.485800+0000","last_undegraded":"2024-09-12T01:33:19.485800+0000","last_fullsized":"2024-09-12T01:33:19.485800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:01.750852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218829+0000","last_change":"2024-09-12T01:33:12.171774+0000","last_active":"2024-09-12T01:33:18.218829+0000","last_peered":"2024-09-12T01:33:18.218829+0000","last_clean":"2024-09-12T01:33:18.218829+0000","last_became_active":"2024-09-12T01:33:12.171472+0000","last_became_peered":"2024-09-12T01:33:12.171472+0000","last_unstale":"2024-09-12T01:33:18.218829+0000","last_undegraded":"2024-09-12T01:33:18.218829+0000","last_fullsized":"2024-09-12T01:33:18.218829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:50.682111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839456+0000","last_change":"2024-09-12T01:33:11.146150+0000","last_active":"2024-09-12T01:33:18.839456+0000","last_peered":"2024-09-12T01:33:18.839456+0000","last_clean":"2024-09-12T01:33:18.839456+0000","last_became_active":"2024-09-12T01:33:11.145690+0000","last_became_peered":"2024-09-12T01:33:11.145690+0000","last_unstale":"2024-09-12T01:33:18.839456+0000","last_undegraded":"2024-09-12T01:33:18.839456+0000","last_fullsized":"2024-09-12T01:33:18.839456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:49.680053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486103+0000","last_change":"2024-09-12T01:33:16.200773+0000","last_active":"2024-09-12T01:33:19.486103+0000","last_peered":"2024-09-12T01:33:19.486103+0000","last_clean":"2024-09-12T01:33:19.486103+0000","last_became_active":"2024-09-12T01:33:16.199757+0000","last_became_peered":"2024-09-12T01:33:16.199757+0000","last_unstale":"2024-09-12T01:33:19.486103+0000","last_undegraded":"2024-09-12T01:33:19.486103+0000","last_fullsized":"2024-09-12T01:33:19.486103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:40.563121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485669+0000","last_change":"2024-09-12T01:33:12.162181+0000","last_active":"2024-09-12T01:33:19.485669+0000","last_peered":"2024-09-12T01:33:19.485669+0000","last_clean":"2024-09-12T01:33:19.485669+0000","last_became_active":"2024-09-12T01:33:12.161900+0000","last_became_peered":"2024-09-12T01:33:12.161900+0000","last_unstale":"2024-09-12T01:33:19.485669+0000","last_undegraded":"2024-09-12T01:33:19.485669+0000","last_fullsized":"2024-09-12T01:33:19.485669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:52.439908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554120+0000","last_change":"2024-09-12T01:33:11.147499+0000","last_active":"2024-09-12T01:33:18.554120+0000","last_peered":"2024-09-12T01:33:18.554120+0000","last_clean":"2024-09-12T01:33:18.554120+0000","last_became_active":"2024-09-12T01:33:11.147186+0000","last_became_peered":"2024-09-12T01:33:11.147186+0000","last_unstale":"2024-09-12T01:33:18.554120+0000","last_undegraded":"2024-09-12T01:33:18.554120+0000","last_fullsized":"2024-09-12T01:33:18.554120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:15.921424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486454+0000","last_change":"2024-09-12T01:33:16.201262+0000","last_active":"2024-09-12T01:33:19.486454+0000","last_peered":"2024-09-12T01:33:19.486454+0000","last_clean":"2024-09-12T01:33:19.486454+0000","last_became_active":"2024-09-12T01:33:16.200850+0000","last_became_peered":"2024-09-12T01:33:16.200850+0000","last_unstale":"2024-09-12T01:33:19.486454+0000","last_undegraded":"2024-09-12T01:33:19.486454+0000","last_fullsized":"2024-09-12T01:33:19.486454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:13.593524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.412948+0000","last_change":"2024-09-12T01:33:12.177601+0000","last_active":"2024-09-12T01:33:19.412948+0000","last_peered":"2024-09-12T01:33:19.412948+0000","last_clean":"2024-09-12T01:33:19.412948+0000","last_became_active":"2024-09-12T01:33:12.177310+0000","last_became_peered":"2024-09-12T01:33:12.177310+0000","last_unstale":"2024-09-12T01:33:19.412948+0000","last_undegraded":"2024-09-12T01:33:19.412948+0000","last_fullsized":"2024-09-12T01:33:19.412948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:59.541178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486689+0000","last_change":"2024-09-12T01:33:11.151233+0000","last_active":"2024-09-12T01:33:19.486689+0000","last_peered":"2024-09-12T01:33:19.486689+0000","last_clean":"2024-09-12T01:33:19.486689+0000","last_became_active":"2024-09-12T01:33:11.150960+0000","last_became_peered":"2024-09-12T01:33:11.150960+0000","last_unstale":"2024-09-12T01:33:19.486689+0000","last_undegraded":"2024-09-12T01:33:19.486689+0000","last_fullsized":"2024-09-12T01:33:19.486689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413558+0000","last_change":"2024-09-12T01:33:16.593340+0000","last_active":"2024-09-12T01:33:19.413558+0000","last_peered":"2024-09-12T01:33:19.413558+0000","last_clean":"2024-09-12T01:33:19.413558+0000","last_became_active":"2024-09-12T01:33:16.592942+0000","last_became_peered":"2024-09-12T01:33:16.592942+0000","last_unstale":"2024-09-12T01:33:19.413558+0000","last_undegraded":"2024-09-12T01:33:19.413558+0000","last_fullsized":"2024-09-12T01:33:19.413558+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:06.180609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216813+0000","last_change":"2024-09-12T01:33:12.175958+0000","last_active":"2024-09-12T01:33:18.216813+0000","last_peered":"2024-09-12T01:33:18.216813+0000","last_clean":"2024-09-12T01:33:18.216813+0000","last_became_active":"2024-09-12T01:33:12.174098+0000","last_became_peered":"2024-09-12T01:33:12.174098+0000","last_unstale":"2024-09-12T01:33:18.216813+0000","last_undegraded":"2024-09-12T01:33:18.216813+0000","last_fullsized":"2024-09-12T01:33:18.216813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839416+0000","last_change":"2024-09-12T01:33:11.148301+0000","last_active":"2024-09-12T01:33:18.839416+0000","last_peered":"2024-09-12T01:33:18.839416+0000","last_clean":"2024-09-12T01:33:18.839416+0000","last_became_active":"2024-09-12T01:33:11.146483+0000","last_became_peered":"2024-09-12T01:33:11.146483+0000","last_unstale":"2024-09-12T01:33:18.839416+0000","last_undegraded":"2024-09-12T01:33:18.839416+0000","last_fullsized":"2024-09-12T01:33:18.839416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:11.527363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486566+0000","last_change":"2024-09-12T01:33:01.096542+0000","last_active":"2024-09-12T01:33:19.486566+0000","last_peered":"2024-09-12T01:33:19.486566+0000","last_clean":"2024-09-12T01:33:19.486566+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:33:19.486566+0000","last_undegraded":"2024-09-12T01:33:19.486566+0000","last_fullsized":"2024-09-12T01:33:19.486566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_clean_scrub_stamp":"2024-09-12T01:32:58.512553+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:48:58.741813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486931+0000","last_change":"2024-09-12T01:33:16.591360+0000","last_active":"2024-09-12T01:33:19.486931+0000","last_peered":"2024-09-12T01:33:19.486931+0000","last_clean":"2024-09-12T01:33:19.486931+0000","last_became_active":"2024-09-12T01:33:16.590884+0000","last_became_peered":"2024-09-12T01:33:16.590884+0000","last_unstale":"2024-09-12T01:33:19.486931+0000","last_undegraded":"2024-09-12T01:33:19.486931+0000","last_fullsized":"2024-09-12T01:33:19.486931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:15.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487399+0000","last_change":"2024-09-12T01:33:12.178384+0000","last_active":"2024-09-12T01:33:19.487399+0000","last_peered":"2024-09-12T01:33:19.487399+0000","last_clean":"2024-09-12T01:33:19.487399+0000","last_became_active":"2024-09-12T01:33:12.178068+0000","last_became_peered":"2024-09-12T01:33:12.178068+0000","last_unstale":"2024-09-12T01:33:19.487399+0000","last_undegraded":"2024-09-12T01:33:19.487399+0000","last_fullsized":"2024-09-12T01:33:19.487399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:51.254891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554226+0000","last_change":"2024-09-12T01:33:11.149796+0000","last_active":"2024-09-12T01:33:18.554226+0000","last_peered":"2024-09-12T01:33:18.554226+0000","last_clean":"2024-09-12T01:33:18.554226+0000","last_became_active":"2024-09-12T01:33:11.149505+0000","last_became_peered":"2024-09-12T01:33:11.149505+0000","last_unstale":"2024-09-12T01:33:18.554226+0000","last_undegraded":"2024-09-12T01:33:18.554226+0000","last_fullsized":"2024-09-12T01:33:18.554226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:24.957500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486471+0000","last_change":"2024-09-12T01:33:16.589125+0000","last_active":"2024-09-12T01:33:19.486471+0000","last_peered":"2024-09-12T01:33:19.486471+0000","last_clean":"2024-09-12T01:33:19.486471+0000","last_became_active":"2024-09-12T01:33:16.588637+0000","last_became_peered":"2024-09-12T01:33:16.588637+0000","last_unstale":"2024-09-12T01:33:19.486471+0000","last_undegraded":"2024-09-12T01:33:19.486471+0000","last_fullsized":"2024-09-12T01:33:19.486471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:06.997893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839688+0000","last_change":"2024-09-12T01:33:12.176967+0000","last_active":"2024-09-12T01:33:18.839688+0000","last_peered":"2024-09-12T01:33:18.839688+0000","last_clean":"2024-09-12T01:33:18.839688+0000","last_became_active":"2024-09-12T01:33:12.176355+0000","last_became_peered":"2024-09-12T01:33:12.176355+0000","last_unstale":"2024-09-12T01:33:18.839688+0000","last_undegraded":"2024-09-12T01:33:18.839688+0000","last_fullsized":"2024-09-12T01:33:18.839688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:57.532127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485317+0000","last_change":"2024-09-12T01:33:11.149279+0000","last_active":"2024-09-12T01:33:19.485317+0000","last_peered":"2024-09-12T01:33:19.485317+0000","last_clean":"2024-09-12T01:33:19.485317+0000","last_became_active":"2024-09-12T01:33:11.148215+0000","last_became_peered":"2024-09-12T01:33:11.148215+0000","last_unstale":"2024-09-12T01:33:19.485317+0000","last_undegraded":"2024-09-12T01:33:19.485317+0000","last_fullsized":"2024-09-12T01:33:19.485317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:49.680053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413355+0000","last_change":"2024-09-12T01:33:16.200672+0000","last_active":"2024-09-12T01:33:19.413355+0000","last_peered":"2024-09-12T01:33:19.413355+0000","last_clean":"2024-09-12T01:33:19.413355+0000","last_became_active":"2024-09-12T01:33:16.200251+0000","last_became_peered":"2024-09-12T01:33:16.200251+0000","last_unstale":"2024-09-12T01:33:19.413355+0000","last_undegraded":"2024-09-12T01:33:19.413355+0000","last_fullsized":"2024-09-12T01:33:19.413355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:37.500171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487810+0000","last_change":"2024-09-12T01:33:12.179113+0000","last_active":"2024-09-12T01:33:19.487810+0000","last_peered":"2024-09-12T01:33:19.487810+0000","last_clean":"2024-09-12T01:33:19.487810+0000","last_became_active":"2024-09-12T01:33:12.178495+0000","last_became_peered":"2024-09-12T01:33:12.178495+0000","last_unstale":"2024-09-12T01:33:19.487810+0000","last_undegraded":"2024-09-12T01:33:19.487810+0000","last_fullsized":"2024-09-12T01:33:19.487810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:51:57.703184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554326+0000","last_change":"2024-09-12T01:33:11.144585+0000","last_active":"2024-09-12T01:33:18.554326+0000","last_peered":"2024-09-12T01:33:18.554326+0000","last_clean":"2024-09-12T01:33:18.554326+0000","last_became_active":"2024-09-12T01:33:11.144277+0000","last_became_peered":"2024-09-12T01:33:11.144277+0000","last_unstale":"2024-09-12T01:33:18.554326+0000","last_undegraded":"2024-09-12T01:33:18.554326+0000","last_fullsized":"2024-09-12T01:33:18.554326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:15.342457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839029+0000","last_change":"2024-09-12T01:33:16.591894+0000","last_active":"2024-09-12T01:33:18.839029+0000","last_peered":"2024-09-12T01:33:18.839029+0000","last_clean":"2024-09-12T01:33:18.839029+0000","last_became_active":"2024-09-12T01:33:16.591452+0000","last_became_peered":"2024-09-12T01:33:16.591452+0000","last_unstale":"2024-09-12T01:33:18.839029+0000","last_undegraded":"2024-09-12T01:33:18.839029+0000","last_fullsized":"2024-09-12T01:33:18.839029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:59.574294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487365+0000","last_change":"2024-09-12T01:33:12.176012+0000","last_active":"2024-09-12T01:33:19.487365+0000","last_peered":"2024-09-12T01:33:19.487365+0000","last_clean":"2024-09-12T01:33:19.487365+0000","last_became_active":"2024-09-12T01:33:12.175729+0000","last_became_peered":"2024-09-12T01:33:12.175729+0000","last_unstale":"2024-09-12T01:33:19.487365+0000","last_undegraded":"2024-09-12T01:33:19.487365+0000","last_fullsized":"2024-09-12T01:33:19.487365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:57.532127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.219006+0000","last_change":"2024-09-12T01:33:11.138837+0000","last_active":"2024-09-12T01:33:18.219006+0000","last_peered":"2024-09-12T01:33:18.219006+0000","last_clean":"2024-09-12T01:33:18.219006+0000","last_became_active":"2024-09-12T01:33:11.138324+0000","last_became_peered":"2024-09-12T01:33:11.138324+0000","last_unstale":"2024-09-12T01:33:18.219006+0000","last_undegraded":"2024-09-12T01:33:18.219006+0000","last_fullsized":"2024-09-12T01:33:18.219006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:48.706430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217732+0000","last_change":"2024-09-12T01:33:16.200428+0000","last_active":"2024-09-12T01:33:18.217732+0000","last_peered":"2024-09-12T01:33:18.217732+0000","last_clean":"2024-09-12T01:33:18.217732+0000","last_became_active":"2024-09-12T01:33:16.199943+0000","last_became_peered":"2024-09-12T01:33:16.199943+0000","last_unstale":"2024-09-12T01:33:18.217732+0000","last_undegraded":"2024-09-12T01:33:18.217732+0000","last_fullsized":"2024-09-12T01:33:18.217732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:15.555695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486512+0000","last_change":"2024-09-12T01:33:12.176546+0000","last_active":"2024-09-12T01:33:19.486512+0000","last_peered":"2024-09-12T01:33:19.486512+0000","last_clean":"2024-09-12T01:33:19.486512+0000","last_became_active":"2024-09-12T01:33:12.176254+0000","last_became_peered":"2024-09-12T01:33:12.176254+0000","last_unstale":"2024-09-12T01:33:19.486512+0000","last_undegraded":"2024-09-12T01:33:19.486512+0000","last_fullsized":"2024-09-12T01:33:19.486512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:24.363602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.553875+0000","last_change":"2024-09-12T01:33:11.143423+0000","last_active":"2024-09-12T01:33:18.553875+0000","last_peered":"2024-09-12T01:33:18.553875+0000","last_clean":"2024-09-12T01:33:18.553875+0000","last_became_active":"2024-09-12T01:33:11.143125+0000","last_became_peered":"2024-09-12T01:33:11.143125+0000","last_unstale":"2024-09-12T01:33:18.553875+0000","last_undegraded":"2024-09-12T01:33:18.553875+0000","last_fullsized":"2024-09-12T01:33:18.553875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:24.005795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413405+0000","last_change":"2024-09-12T01:33:16.190124+0000","last_active":"2024-09-12T01:33:19.413405+0000","last_peered":"2024-09-12T01:33:19.413405+0000","last_clean":"2024-09-12T01:33:19.413405+0000","last_became_active":"2024-09-12T01:33:16.189728+0000","last_became_peered":"2024-09-12T01:33:16.189728+0000","last_unstale":"2024-09-12T01:33:19.413405+0000","last_undegraded":"2024-09-12T01:33:19.413405+0000","last_fullsized":"2024-09-12T01:33:19.413405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:00.559557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486163+0000","last_change":"2024-09-12T01:33:12.173415+0000","last_active":"2024-09-12T01:33:19.486163+0000","last_peered":"2024-09-12T01:33:19.486163+0000","last_clean":"2024-09-12T01:33:19.486163+0000","last_became_active":"2024-09-12T01:33:12.173117+0000","last_became_peered":"2024-09-12T01:33:12.173117+0000","last_unstale":"2024-09-12T01:33:19.486163+0000","last_undegraded":"2024-09-12T01:33:19.486163+0000","last_fullsized":"2024-09-12T01:33:19.486163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:54.679349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486025+0000","last_change":"2024-09-12T01:33:11.140718+0000","last_active":"2024-09-12T01:33:19.486025+0000","last_peered":"2024-09-12T01:33:19.486025+0000","last_clean":"2024-09-12T01:33:19.486025+0000","last_became_active":"2024-09-12T01:33:11.140321+0000","last_became_peered":"2024-09-12T01:33:11.140321+0000","last_unstale":"2024-09-12T01:33:19.486025+0000","last_undegraded":"2024-09-12T01:33:19.486025+0000","last_fullsized":"2024-09-12T01:33:19.486025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:22.885518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217295+0000","last_change":"2024-09-12T01:33:16.594875+0000","last_active":"2024-09-12T01:33:18.217295+0000","last_peered":"2024-09-12T01:33:18.217295+0000","last_clean":"2024-09-12T01:33:18.217295+0000","last_became_active":"2024-09-12T01:33:16.592733+0000","last_became_peered":"2024-09-12T01:33:16.592733+0000","last_unstale":"2024-09-12T01:33:18.217295+0000","last_undegraded":"2024-09-12T01:33:18.217295+0000","last_fullsized":"2024-09-12T01:33:18.217295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:39.080379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486853+0000","last_change":"2024-09-12T01:33:12.175983+0000","last_active":"2024-09-12T01:33:19.486853+0000","last_peered":"2024-09-12T01:33:19.486853+0000","last_clean":"2024-09-12T01:33:19.486853+0000","last_became_active":"2024-09-12T01:33:12.175638+0000","last_became_peered":"2024-09-12T01:33:12.175638+0000","last_unstale":"2024-09-12T01:33:19.486853+0000","last_undegraded":"2024-09-12T01:33:19.486853+0000","last_fullsized":"2024-09-12T01:33:19.486853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:48.292206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839432+0000","last_change":"2024-09-12T01:33:11.148832+0000","last_active":"2024-09-12T01:33:18.839432+0000","last_peered":"2024-09-12T01:33:18.839432+0000","last_clean":"2024-09-12T01:33:18.839432+0000","last_became_active":"2024-09-12T01:33:11.148519+0000","last_became_peered":"2024-09-12T01:33:11.148519+0000","last_unstale":"2024-09-12T01:33:18.839432+0000","last_undegraded":"2024-09-12T01:33:18.839432+0000","last_fullsized":"2024-09-12T01:33:18.839432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217452+0000","last_change":"2024-09-12T01:33:16.588958+0000","last_active":"2024-09-12T01:33:18.217452+0000","last_peered":"2024-09-12T01:33:18.217452+0000","last_clean":"2024-09-12T01:33:18.217452+0000","last_became_active":"2024-09-12T01:33:16.588510+0000","last_became_peered":"2024-09-12T01:33:16.588510+0000","last_unstale":"2024-09-12T01:33:18.217452+0000","last_undegraded":"2024-09-12T01:33:18.217452+0000","last_fullsized":"2024-09-12T01:33:18.217452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:00.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413221+0000","last_change":"2024-09-12T01:33:12.184393+0000","last_active":"2024-09-12T01:33:19.413221+0000","last_peered":"2024-09-12T01:33:19.413221+0000","last_clean":"2024-09-12T01:33:19.413221+0000","last_became_active":"2024-09-12T01:33:12.184103+0000","last_became_peered":"2024-09-12T01:33:12.184103+0000","last_unstale":"2024-09-12T01:33:19.413221+0000","last_undegraded":"2024-09-12T01:33:19.413221+0000","last_fullsized":"2024-09-12T01:33:19.413221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:36.026056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.412683+0000","last_change":"2024-09-12T01:33:11.134020+0000","last_active":"2024-09-12T01:33:19.412683+0000","last_peered":"2024-09-12T01:33:19.412683+0000","last_clean":"2024-09-12T01:33:19.412683+0000","last_became_active":"2024-09-12T01:33:11.133717+0000","last_became_peered":"2024-09-12T01:33:11.133717+0000","last_unstale":"2024-09-12T01:33:19.412683+0000","last_undegraded":"2024-09-12T01:33:19.412683+0000","last_fullsized":"2024-09-12T01:33:19.412683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:20.901290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839735+0000","last_change":"2024-09-12T01:33:16.593051+0000","last_active":"2024-09-12T01:33:18.839735+0000","last_peered":"2024-09-12T01:33:18.839735+0000","last_clean":"2024-09-12T01:33:18.839735+0000","last_became_active":"2024-09-12T01:33:16.592513+0000","last_became_peered":"2024-09-12T01:33:16.592513+0000","last_unstale":"2024-09-12T01:33:18.839735+0000","last_undegraded":"2024-09-12T01:33:18.839735+0000","last_fullsized":"2024-09-12T01:33:18.839735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:35.903448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487266+0000","last_change":"2024-09-12T01:33:12.178596+0000","last_active":"2024-09-12T01:33:19.487266+0000","last_peered":"2024-09-12T01:33:19.487266+0000","last_clean":"2024-09-12T01:33:19.487266+0000","last_became_active":"2024-09-12T01:33:12.176316+0000","last_became_peered":"2024-09-12T01:33:12.176316+0000","last_unstale":"2024-09-12T01:33:19.487266+0000","last_undegraded":"2024-09-12T01:33:19.487266+0000","last_fullsized":"2024-09-12T01:33:19.487266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:24.134321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:21.709458+0000","last_change":"2024-09-12T01:33:11.147275+0000","last_active":"2024-09-12T01:33:21.709458+0000","last_peered":"2024-09-12T01:33:21.709458+0000","last_clean":"2024-09-12T01:33:21.709458+0000","last_became_active":"2024-09-12T01:33:11.146962+0000","last_became_peered":"2024-09-12T01:33:11.146962+0000","last_unstale":"2024-09-12T01:33:21.709458+0000","last_undegraded":"2024-09-12T01:33:21.709458+0000","last_fullsized":"2024-09-12T01:33:21.709458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:11.527363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485898+0000","last_change":"2024-09-12T01:33:16.202736+0000","last_active":"2024-09-12T01:33:19.485898+0000","last_peered":"2024-09-12T01:33:19.485898+0000","last_clean":"2024-09-12T01:33:19.485898+0000","last_became_active":"2024-09-12T01:33:16.202282+0000","last_became_peered":"2024-09-12T01:33:16.202282+0000","last_unstale":"2024-09-12T01:33:19.485898+0000","last_undegraded":"2024-09-12T01:33:19.485898+0000","last_fullsized":"2024-09-12T01:33:19.485898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:28.321407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218580+0000","last_change":"2024-09-12T01:33:12.173488+0000","last_active":"2024-09-12T01:33:18.218580+0000","last_peered":"2024-09-12T01:33:18.218580+0000","last_clean":"2024-09-12T01:33:18.218580+0000","last_became_active":"2024-09-12T01:33:12.173187+0000","last_became_peered":"2024-09-12T01:33:12.173187+0000","last_unstale":"2024-09-12T01:33:18.218580+0000","last_undegraded":"2024-09-12T01:33:18.218580+0000","last_fullsized":"2024-09-12T01:33:18.218580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:27.581552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485330+0000","last_change":"2024-09-12T01:33:11.149565+0000","last_active":"2024-09-12T01:33:19.485330+0000","last_peered":"2024-09-12T01:33:19.485330+0000","last_clean":"2024-09-12T01:33:19.485330+0000","last_became_active":"2024-09-12T01:33:11.149034+0000","last_became_peered":"2024-09-12T01:33:11.149034+0000","last_unstale":"2024-09-12T01:33:19.485330+0000","last_undegraded":"2024-09-12T01:33:19.485330+0000","last_fullsized":"2024-09-12T01:33:19.485330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218212+0000","last_change":"2024-09-12T01:33:16.202414+0000","last_active":"2024-09-12T01:33:18.218212+0000","last_peered":"2024-09-12T01:33:18.218212+0000","last_clean":"2024-09-12T01:33:18.218212+0000","last_became_active":"2024-09-12T01:33:16.200143+0000","last_became_peered":"2024-09-12T01:33:16.200143+0000","last_unstale":"2024-09-12T01:33:18.218212+0000","last_undegraded":"2024-09-12T01:33:18.218212+0000","last_fullsized":"2024-09-12T01:33:18.218212+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:14.088579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413595+0000","last_change":"2024-09-12T01:33:12.174901+0000","last_active":"2024-09-12T01:33:19.413595+0000","last_peered":"2024-09-12T01:33:19.413595+0000","last_clean":"2024-09-12T01:33:19.413595+0000","last_became_active":"2024-09-12T01:33:12.174638+0000","last_became_peered":"2024-09-12T01:33:12.174638+0000","last_unstale":"2024-09-12T01:33:19.413595+0000","last_undegraded":"2024-09-12T01:33:19.413595+0000","last_fullsized":"2024-09-12T01:33:19.413595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:54.169299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839752+0000","last_change":"2024-09-12T01:33:11.147387+0000","last_active":"2024-09-12T01:33:18.839752+0000","last_peered":"2024-09-12T01:33:18.839752+0000","last_clean":"2024-09-12T01:33:18.839752+0000","last_became_active":"2024-09-12T01:33:11.147110+0000","last_became_peered":"2024-09-12T01:33:11.147110+0000","last_unstale":"2024-09-12T01:33:18.839752+0000","last_undegraded":"2024-09-12T01:33:18.839752+0000","last_fullsized":"2024-09-12T01:33:18.839752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:55.391944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216657+0000","last_change":"2024-09-12T01:33:16.191800+0000","last_active":"2024-09-12T01:33:18.216657+0000","last_peered":"2024-09-12T01:33:18.216657+0000","last_clean":"2024-09-12T01:33:18.216657+0000","last_became_active":"2024-09-12T01:33:16.190448+0000","last_became_peered":"2024-09-12T01:33:16.190448+0000","last_unstale":"2024-09-12T01:33:18.216657+0000","last_undegraded":"2024-09-12T01:33:18.216657+0000","last_fullsized":"2024-09-12T01:33:18.216657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:47.099767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839819+0000","last_change":"2024-09-12T01:33:12.178000+0000","last_active":"2024-09-12T01:33:18.839819+0000","last_peered":"2024-09-12T01:33:18.839819+0000","last_clean":"2024-09-12T01:33:18.839819+0000","last_became_active":"2024-09-12T01:33:12.177703+0000","last_became_peered":"2024-09-12T01:33:12.177703+0000","last_unstale":"2024-09-12T01:33:18.839819+0000","last_undegraded":"2024-09-12T01:33:18.839819+0000","last_fullsized":"2024-09-12T01:33:18.839819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:56.959747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485907+0000","last_change":"2024-09-12T01:33:11.149688+0000","last_active":"2024-09-12T01:33:19.485907+0000","last_peered":"2024-09-12T01:33:19.485907+0000","last_clean":"2024-09-12T01:33:19.485907+0000","last_became_active":"2024-09-12T01:33:11.149363+0000","last_became_peered":"2024-09-12T01:33:11.149363+0000","last_unstale":"2024-09-12T01:33:19.485907+0000","last_undegraded":"2024-09-12T01:33:19.485907+0000","last_fullsized":"2024-09-12T01:33:19.485907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216625+0000","last_change":"2024-09-12T01:33:16.592951+0000","last_active":"2024-09-12T01:33:18.216625+0000","last_peered":"2024-09-12T01:33:18.216625+0000","last_clean":"2024-09-12T01:33:18.216625+0000","last_became_active":"2024-09-12T01:33:16.592558+0000","last_became_peered":"2024-09-12T01:33:16.592558+0000","last_unstale":"2024-09-12T01:33:18.216625+0000","last_undegraded":"2024-09-12T01:33:18.216625+0000","last_fullsized":"2024-09-12T01:33:18.216625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:29.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413385+0000","last_change":"2024-09-12T01:33:16.594318+0000","last_active":"2024-09-12T01:33:19.413385+0000","last_peered":"2024-09-12T01:33:19.413385+0000","last_clean":"2024-09-12T01:33:19.413385+0000","last_became_active":"2024-09-12T01:33:16.593834+0000","last_became_peered":"2024-09-12T01:33:16.593834+0000","last_unstale":"2024-09-12T01:33:19.413385+0000","last_undegraded":"2024-09-12T01:33:19.413385+0000","last_fullsized":"2024-09-12T01:33:19.413385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:11.126504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485462+0000","last_change":"2024-09-12T01:33:12.171282+0000","last_active":"2024-09-12T01:33:19.485462+0000","last_peered":"2024-09-12T01:33:19.485462+0000","last_clean":"2024-09-12T01:33:19.485462+0000","last_became_active":"2024-09-12T01:33:12.170988+0000","last_became_peered":"2024-09-12T01:33:12.170988+0000","last_unstale":"2024-09-12T01:33:19.485462+0000","last_undegraded":"2024-09-12T01:33:19.485462+0000","last_fullsized":"2024-09-12T01:33:19.485462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:25.007983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839177+0000","last_change":"2024-09-12T01:33:11.146497+0000","last_active":"2024-09-12T01:33:18.839177+0000","last_peered":"2024-09-12T01:33:18.839177+0000","last_clean":"2024-09-12T01:33:18.839177+0000","last_became_active":"2024-09-12T01:33:11.146184+0000","last_became_peered":"2024-09-12T01:33:11.146184+0000","last_unstale":"2024-09-12T01:33:18.839177+0000","last_undegraded":"2024-09-12T01:33:18.839177+0000","last_fullsized":"2024-09-12T01:33:18.839177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:22.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487513+0000","last_change":"2024-09-12T01:33:16.202584+0000","last_active":"2024-09-12T01:33:19.487513+0000","last_peered":"2024-09-12T01:33:19.487513+0000","last_clean":"2024-09-12T01:33:19.487513+0000","last_became_active":"2024-09-12T01:33:16.202137+0000","last_became_peered":"2024-09-12T01:33:16.202137+0000","last_unstale":"2024-09-12T01:33:19.487513+0000","last_undegraded":"2024-09-12T01:33:19.487513+0000","last_fullsized":"2024-09-12T01:33:19.487513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:39.155565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216613+0000","last_change":"2024-09-12T01:33:12.169183+0000","last_active":"2024-09-12T01:33:18.216613+0000","last_peered":"2024-09-12T01:33:18.216613+0000","last_clean":"2024-09-12T01:33:18.216613+0000","last_became_active":"2024-09-12T01:33:12.168803+0000","last_became_peered":"2024-09-12T01:33:12.168803+0000","last_unstale":"2024-09-12T01:33:18.216613+0000","last_undegraded":"2024-09-12T01:33:18.216613+0000","last_fullsized":"2024-09-12T01:33:18.216613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:25.007983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216934+0000","last_change":"2024-09-12T01:33:11.144264+0000","last_active":"2024-09-12T01:33:18.216934+0000","last_peered":"2024-09-12T01:33:18.216934+0000","last_clean":"2024-09-12T01:33:18.216934+0000","last_became_active":"2024-09-12T01:33:11.143993+0000","last_became_peered":"2024-09-12T01:33:11.143993+0000","last_unstale":"2024-09-12T01:33:18.216934+0000","last_undegraded":"2024-09-12T01:33:18.216934+0000","last_fullsized":"2024-09-12T01:33:18.216934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:02:33.335627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485661+0000","last_change":"2024-09-12T01:33:11.143410+0000","last_active":"2024-09-12T01:33:19.485661+0000","last_peered":"2024-09-12T01:33:19.485661+0000","last_clean":"2024-09-12T01:33:19.485661+0000","last_became_active":"2024-09-12T01:33:11.143121+0000","last_became_peered":"2024-09-12T01:33:11.143121+0000","last_unstale":"2024-09-12T01:33:19.485661+0000","last_undegraded":"2024-09-12T01:33:19.485661+0000","last_fullsized":"2024-09-12T01:33:19.485661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:49.000466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413493+0000","last_change":"2024-09-12T01:33:12.176891+0000","last_active":"2024-09-12T01:33:19.413493+0000","last_peered":"2024-09-12T01:33:19.413493+0000","last_clean":"2024-09-12T01:33:19.413493+0000","last_became_active":"2024-09-12T01:33:12.176573+0000","last_became_peered":"2024-09-12T01:33:12.176573+0000","last_unstale":"2024-09-12T01:33:19.413493+0000","last_undegraded":"2024-09-12T01:33:19.413493+0000","last_fullsized":"2024-09-12T01:33:19.413493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:47.157354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554126+0000","last_change":"2024-09-12T01:33:16.195123+0000","last_active":"2024-09-12T01:33:18.554126+0000","last_peered":"2024-09-12T01:33:18.554126+0000","last_clean":"2024-09-12T01:33:18.554126+0000","last_became_active":"2024-09-12T01:33:16.194690+0000","last_became_peered":"2024-09-12T01:33:16.194690+0000","last_unstale":"2024-09-12T01:33:18.554126+0000","last_undegraded":"2024-09-12T01:33:18.554126+0000","last_fullsized":"2024-09-12T01:33:18.554126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:14.296077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217027+0000","last_change":"2024-09-12T01:33:11.145695+0000","last_active":"2024-09-12T01:33:18.217027+0000","last_peered":"2024-09-12T01:33:18.217027+0000","last_clean":"2024-09-12T01:33:18.217027+0000","last_became_active":"2024-09-12T01:33:11.145391+0000","last_became_peered":"2024-09-12T01:33:11.145391+0000","last_unstale":"2024-09-12T01:33:18.217027+0000","last_undegraded":"2024-09-12T01:33:18.217027+0000","last_fullsized":"2024-09-12T01:33:18.217027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:10.119806+0000","last_clean_scrub_stamp":"2024-09-12T01:33:10.119806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:47.695596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218383+0000","last_change":"2024-09-12T01:33:12.164156+0000","last_active":"2024-09-12T01:33:18.218383+0000","last_peered":"2024-09-12T01:33:18.218383+0000","last_clean":"2024-09-12T01:33:18.218383+0000","last_became_active":"2024-09-12T01:33:12.160798+0000","last_became_peered":"2024-09-12T01:33:12.160798+0000","last_unstale":"2024-09-12T01:33:18.218383+0000","last_undegraded":"2024-09-12T01:33:18.218383+0000","last_fullsized":"2024-09-12T01:33:18.218383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:56.394003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217262+0000","last_change":"2024-09-12T01:33:16.198377+0000","last_active":"2024-09-12T01:33:18.217262+0000","last_peered":"2024-09-12T01:33:18.217262+0000","last_clean":"2024-09-12T01:33:18.217262+0000","last_became_active":"2024-09-12T01:33:16.197913+0000","last_became_peered":"2024-09-12T01:33:16.197913+0000","last_unstale":"2024-09-12T01:33:18.217262+0000","last_undegraded":"2024-09-12T01:33:18.217262+0000","last_fullsized":"2024-09-12T01:33:18.217262+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:15.169533+0000","last_clean_scrub_stamp":"2024-09-12T01:33:15.169533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:56:05.124457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486041+0000","last_change":"2024-09-12T01:33:12.176137+0000","last_active":"2024-09-12T01:33:19.486041+0000","last_peered":"2024-09-12T01:33:19.486041+0000","last_clean":"2024-09-12T01:33:19.486041+0000","last_became_active":"2024-09-12T01:33:12.175840+0000","last_became_peered":"2024-09-12T01:33:12.175840+0000","last_unstale":"2024-09-12T01:33:19.486041+0000","last_undegraded":"2024-09-12T01:33:19.486041+0000","last_fullsized":"2024-09-12T01:33:19.486041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:38.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554012+0000","last_change":"2024-09-12T01:33:12.168134+0000","last_active":"2024-09-12T01:33:18.554012+0000","last_peered":"2024-09-12T01:33:18.554012+0000","last_clean":"2024-09-12T01:33:18.554012+0000","last_became_active":"2024-09-12T01:33:12.167844+0000","last_became_peered":"2024-09-12T01:33:12.167844+0000","last_unstale":"2024-09-12T01:33:18.554012+0000","last_undegraded":"2024-09-12T01:33:18.554012+0000","last_fullsized":"2024-09-12T01:33:18.554012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:06.175990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839339+0000","last_change":"2024-09-12T01:33:12.183112+0000","last_active":"2024-09-12T01:33:18.839339+0000","last_peered":"2024-09-12T01:33:18.839339+0000","last_clean":"2024-09-12T01:33:18.839339+0000","last_became_active":"2024-09-12T01:33:12.182858+0000","last_became_peered":"2024-09-12T01:33:12.182858+0000","last_unstale":"2024-09-12T01:33:18.839339+0000","last_undegraded":"2024-09-12T01:33:18.839339+0000","last_fullsized":"2024-09-12T01:33:18.839339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:11.508027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413461+0000","last_change":"2024-09-12T01:33:12.181683+0000","last_active":"2024-09-12T01:33:19.413461+0000","last_peered":"2024-09-12T01:33:19.413461+0000","last_clean":"2024-09-12T01:33:19.413461+0000","last_became_active":"2024-09-12T01:33:12.181383+0000","last_became_peered":"2024-09-12T01:33:12.181383+0000","last_unstale":"2024-09-12T01:33:19.413461+0000","last_undegraded":"2024-09-12T01:33:19.413461+0000","last_fullsized":"2024-09-12T01:33:19.413461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:00.865640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217578+0000","last_change":"2024-09-12T01:33:12.178479+0000","last_active":"2024-09-12T01:33:18.217578+0000","last_peered":"2024-09-12T01:33:18.217578+0000","last_clean":"2024-09-12T01:33:18.217578+0000","last_became_active":"2024-09-12T01:33:12.178198+0000","last_became_peered":"2024-09-12T01:33:12.178198+0000","last_unstale":"2024-09-12T01:33:18.217578+0000","last_undegraded":"2024-09-12T01:33:18.217578+0000","last_fullsized":"2024-09-12T01:33:18.217578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:53.398489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485933+0000","last_change":"2024-09-12T01:33:12.168261+0000","last_active":"2024-09-12T01:33:19.485933+0000","last_peered":"2024-09-12T01:33:19.485933+0000","last_clean":"2024-09-12T01:33:19.485933+0000","last_became_active":"2024-09-12T01:33:12.167980+0000","last_became_peered":"2024-09-12T01:33:12.167980+0000","last_unstale":"2024-09-12T01:33:19.485933+0000","last_undegraded":"2024-09-12T01:33:19.485933+0000","last_fullsized":"2024-09-12T01:33:19.485933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:02.393918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487377+0000","last_change":"2024-09-12T01:33:12.186333+0000","last_active":"2024-09-12T01:33:19.487377+0000","last_peered":"2024-09-12T01:33:19.487377+0000","last_clean":"2024-09-12T01:33:19.487377+0000","last_became_active":"2024-09-12T01:33:12.186047+0000","last_became_peered":"2024-09-12T01:33:12.186047+0000","last_unstale":"2024-09-12T01:33:19.487377+0000","last_undegraded":"2024-09-12T01:33:19.487377+0000","last_fullsized":"2024-09-12T01:33:19.487377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:53.398489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.217203+0000","last_change":"2024-09-12T01:33:12.177911+0000","last_active":"2024-09-12T01:33:18.217203+0000","last_peered":"2024-09-12T01:33:18.217203+0000","last_clean":"2024-09-12T01:33:18.217203+0000","last_became_active":"2024-09-12T01:33:12.177625+0000","last_became_peered":"2024-09-12T01:33:12.177625+0000","last_unstale":"2024-09-12T01:33:18.217203+0000","last_undegraded":"2024-09-12T01:33:18.217203+0000","last_fullsized":"2024-09-12T01:33:18.217203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:02.393918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839173+0000","last_change":"2024-09-12T01:33:12.183935+0000","last_active":"2024-09-12T01:33:18.839173+0000","last_peered":"2024-09-12T01:33:18.839173+0000","last_clean":"2024-09-12T01:33:18.839173+0000","last_became_active":"2024-09-12T01:33:12.183620+0000","last_became_peered":"2024-09-12T01:33:12.183620+0000","last_unstale":"2024-09-12T01:33:18.839173+0000","last_undegraded":"2024-09-12T01:33:18.839173+0000","last_fullsized":"2024-09-12T01:33:18.839173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:02.950225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216744+0000","last_change":"2024-09-12T01:33:12.179130+0000","last_active":"2024-09-12T01:33:18.216744+0000","last_peered":"2024-09-12T01:33:18.216744+0000","last_clean":"2024-09-12T01:33:18.216744+0000","last_became_active":"2024-09-12T01:33:12.177141+0000","last_became_peered":"2024-09-12T01:33:12.177141+0000","last_unstale":"2024-09-12T01:33:18.216744+0000","last_undegraded":"2024-09-12T01:33:18.216744+0000","last_fullsized":"2024-09-12T01:33:18.216744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:24.134321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839306+0000","last_change":"2024-09-12T01:33:12.175899+0000","last_active":"2024-09-12T01:33:18.839306+0000","last_peered":"2024-09-12T01:33:18.839306+0000","last_clean":"2024-09-12T01:33:18.839306+0000","last_became_active":"2024-09-12T01:33:12.175594+0000","last_became_peered":"2024-09-12T01:33:12.175594+0000","last_unstale":"2024-09-12T01:33:18.839306+0000","last_undegraded":"2024-09-12T01:33:18.839306+0000","last_fullsized":"2024-09-12T01:33:18.839306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:51.254891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413194+0000","last_change":"2024-09-12T01:33:12.185737+0000","last_active":"2024-09-12T01:33:19.413194+0000","last_peered":"2024-09-12T01:33:19.413194+0000","last_clean":"2024-09-12T01:33:19.413194+0000","last_became_active":"2024-09-12T01:33:12.185489+0000","last_became_peered":"2024-09-12T01:33:12.185489+0000","last_unstale":"2024-09-12T01:33:19.413194+0000","last_undegraded":"2024-09-12T01:33:19.413194+0000","last_fullsized":"2024-09-12T01:33:19.413194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:05.301039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486993+0000","last_change":"2024-09-12T01:33:12.183324+0000","last_active":"2024-09-12T01:33:19.486993+0000","last_peered":"2024-09-12T01:33:19.486993+0000","last_clean":"2024-09-12T01:33:19.486993+0000","last_became_active":"2024-09-12T01:33:12.182857+0000","last_became_peered":"2024-09-12T01:33:12.182857+0000","last_unstale":"2024-09-12T01:33:19.486993+0000","last_undegraded":"2024-09-12T01:33:19.486993+0000","last_fullsized":"2024-09-12T01:33:19.486993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:52.439908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487522+0000","last_change":"2024-09-12T01:33:12.177539+0000","last_active":"2024-09-12T01:33:19.487522+0000","last_peered":"2024-09-12T01:33:19.487522+0000","last_clean":"2024-09-12T01:33:19.487522+0000","last_became_active":"2024-09-12T01:33:12.173980+0000","last_became_peered":"2024-09-12T01:33:12.173980+0000","last_unstale":"2024-09-12T01:33:19.487522+0000","last_undegraded":"2024-09-12T01:33:19.487522+0000","last_fullsized":"2024-09-12T01:33:19.487522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:11.508027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.554334+0000","last_change":"2024-09-12T01:33:12.179198+0000","last_active":"2024-09-12T01:33:18.554334+0000","last_peered":"2024-09-12T01:33:18.554334+0000","last_clean":"2024-09-12T01:33:18.554334+0000","last_became_active":"2024-09-12T01:33:12.178859+0000","last_became_peered":"2024-09-12T01:33:12.178859+0000","last_unstale":"2024-09-12T01:33:18.554334+0000","last_undegraded":"2024-09-12T01:33:18.554334+0000","last_fullsized":"2024-09-12T01:33:18.554334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:46.228922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839997+0000","last_change":"2024-09-12T01:33:12.179001+0000","last_active":"2024-09-12T01:33:18.839997+0000","last_peered":"2024-09-12T01:33:18.839997+0000","last_clean":"2024-09-12T01:33:18.839997+0000","last_became_active":"2024-09-12T01:33:12.178298+0000","last_became_peered":"2024-09-12T01:33:12.178298+0000","last_unstale":"2024-09-12T01:33:18.839997+0000","last_undegraded":"2024-09-12T01:33:18.839997+0000","last_fullsized":"2024-09-12T01:33:18.839997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:35.107896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839003+0000","last_change":"2024-09-12T01:33:12.169863+0000","last_active":"2024-09-12T01:33:18.839003+0000","last_peered":"2024-09-12T01:33:18.839003+0000","last_clean":"2024-09-12T01:33:18.839003+0000","last_became_active":"2024-09-12T01:33:12.169441+0000","last_became_peered":"2024-09-12T01:33:12.169441+0000","last_unstale":"2024-09-12T01:33:18.839003+0000","last_undegraded":"2024-09-12T01:33:18.839003+0000","last_fullsized":"2024-09-12T01:33:18.839003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.485524+0000","last_change":"2024-09-12T01:33:12.160854+0000","last_active":"2024-09-12T01:33:19.485524+0000","last_peered":"2024-09-12T01:33:19.485524+0000","last_clean":"2024-09-12T01:33:19.485524+0000","last_became_active":"2024-09-12T01:33:12.160512+0000","last_became_peered":"2024-09-12T01:33:12.160512+0000","last_unstale":"2024-09-12T01:33:19.485524+0000","last_undegraded":"2024-09-12T01:33:19.485524+0000","last_fullsized":"2024-09-12T01:33:19.485524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:38.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413165+0000","last_change":"2024-09-12T01:33:12.172031+0000","last_active":"2024-09-12T01:33:19.413165+0000","last_peered":"2024-09-12T01:33:19.413165+0000","last_clean":"2024-09-12T01:33:19.413165+0000","last_became_active":"2024-09-12T01:33:12.171730+0000","last_became_peered":"2024-09-12T01:33:12.171730+0000","last_unstale":"2024-09-12T01:33:19.413165+0000","last_undegraded":"2024-09-12T01:33:19.413165+0000","last_fullsized":"2024-09-12T01:33:19.413165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:10.248408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216814+0000","last_change":"2024-09-12T01:33:12.173718+0000","last_active":"2024-09-12T01:33:18.216814+0000","last_peered":"2024-09-12T01:33:18.216814+0000","last_clean":"2024-09-12T01:33:18.216814+0000","last_became_active":"2024-09-12T01:33:12.171379+0000","last_became_peered":"2024-09-12T01:33:12.171379+0000","last_unstale":"2024-09-12T01:33:18.216814+0000","last_undegraded":"2024-09-12T01:33:18.216814+0000","last_fullsized":"2024-09-12T01:33:18.216814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:52.439908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.218338+0000","last_change":"2024-09-12T01:33:12.174895+0000","last_active":"2024-09-12T01:33:18.218338+0000","last_peered":"2024-09-12T01:33:18.218338+0000","last_clean":"2024-09-12T01:33:18.218338+0000","last_became_active":"2024-09-12T01:33:12.174591+0000","last_became_peered":"2024-09-12T01:33:12.174591+0000","last_unstale":"2024-09-12T01:33:18.218338+0000","last_undegraded":"2024-09-12T01:33:18.218338+0000","last_fullsized":"2024-09-12T01:33:18.218338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:57.102634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486963+0000","last_change":"2024-09-12T01:33:12.181588+0000","last_active":"2024-09-12T01:33:19.486963+0000","last_peered":"2024-09-12T01:33:19.486963+0000","last_clean":"2024-09-12T01:33:19.486963+0000","last_became_active":"2024-09-12T01:33:12.181293+0000","last_became_peered":"2024-09-12T01:33:12.181293+0000","last_unstale":"2024-09-12T01:33:19.486963+0000","last_undegraded":"2024-09-12T01:33:19.486963+0000","last_fullsized":"2024-09-12T01:33:19.486963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:02.393918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487308+0000","last_change":"2024-09-12T01:33:12.175792+0000","last_active":"2024-09-12T01:33:19.487308+0000","last_peered":"2024-09-12T01:33:19.487308+0000","last_clean":"2024-09-12T01:33:19.487308+0000","last_became_active":"2024-09-12T01:33:12.175332+0000","last_became_peered":"2024-09-12T01:33:12.175332+0000","last_unstale":"2024-09-12T01:33:19.487308+0000","last_undegraded":"2024-09-12T01:33:19.487308+0000","last_fullsized":"2024-09-12T01:33:19.487308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487796+0000","last_change":"2024-09-12T01:33:12.178831+0000","last_active":"2024-09-12T01:33:19.487796+0000","last_peered":"2024-09-12T01:33:19.487796+0000","last_clean":"2024-09-12T01:33:19.487796+0000","last_became_active":"2024-09-12T01:33:12.178116+0000","last_became_peered":"2024-09-12T01:33:12.178116+0000","last_unstale":"2024-09-12T01:33:19.487796+0000","last_undegraded":"2024-09-12T01:33:19.487796+0000","last_fullsized":"2024-09-12T01:33:19.487796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:35.107896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.216514+0000","last_change":"2024-09-12T01:33:12.172225+0000","last_active":"2024-09-12T01:33:18.216514+0000","last_peered":"2024-09-12T01:33:18.216514+0000","last_clean":"2024-09-12T01:33:18.216514+0000","last_became_active":"2024-09-12T01:33:12.171933+0000","last_became_peered":"2024-09-12T01:33:12.171933+0000","last_unstale":"2024-09-12T01:33:18.216514+0000","last_undegraded":"2024-09-12T01:33:18.216514+0000","last_fullsized":"2024-09-12T01:33:18.216514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:20.867318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486602+0000","last_change":"2024-09-12T01:33:12.173395+0000","last_active":"2024-09-12T01:33:19.486602+0000","last_peered":"2024-09-12T01:33:19.486602+0000","last_clean":"2024-09-12T01:33:19.486602+0000","last_became_active":"2024-09-12T01:33:12.173090+0000","last_became_peered":"2024-09-12T01:33:12.173090+0000","last_unstale":"2024-09-12T01:33:19.486602+0000","last_undegraded":"2024-09-12T01:33:19.486602+0000","last_fullsized":"2024-09-12T01:33:19.486602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:46.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839213+0000","last_change":"2024-09-12T01:33:12.169604+0000","last_active":"2024-09-12T01:33:18.839213+0000","last_peered":"2024-09-12T01:33:18.839213+0000","last_clean":"2024-09-12T01:33:18.839213+0000","last_became_active":"2024-09-12T01:33:12.169258+0000","last_became_peered":"2024-09-12T01:33:12.169258+0000","last_unstale":"2024-09-12T01:33:18.839213+0000","last_undegraded":"2024-09-12T01:33:18.839213+0000","last_fullsized":"2024-09-12T01:33:18.839213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:46.612807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486571+0000","last_change":"2024-09-12T01:33:12.171033+0000","last_active":"2024-09-12T01:33:19.486571+0000","last_peered":"2024-09-12T01:33:19.486571+0000","last_clean":"2024-09-12T01:33:19.486571+0000","last_became_active":"2024-09-12T01:33:12.170708+0000","last_became_peered":"2024-09-12T01:33:12.170708+0000","last_unstale":"2024-09-12T01:33:19.486571+0000","last_undegraded":"2024-09-12T01:33:19.486571+0000","last_fullsized":"2024-09-12T01:33:19.486571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:56.959747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:18.839539+0000","last_change":"2024-09-12T01:33:12.182780+0000","last_active":"2024-09-12T01:33:18.839539+0000","last_peered":"2024-09-12T01:33:18.839539+0000","last_clean":"2024-09-12T01:33:18.839539+0000","last_became_active":"2024-09-12T01:33:12.182454+0000","last_became_peered":"2024-09-12T01:33:12.182454+0000","last_unstale":"2024-09-12T01:33:18.839539+0000","last_undegraded":"2024-09-12T01:33:18.839539+0000","last_fullsized":"2024-09-12T01:33:18.839539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:53.398489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.413295+0000","last_change":"2024-09-12T01:33:12.180723+0000","last_active":"2024-09-12T01:33:19.413295+0000","last_peered":"2024-09-12T01:33:19.413295+0000","last_clean":"2024-09-12T01:33:19.413295+0000","last_became_active":"2024-09-12T01:33:12.180414+0000","last_became_peered":"2024-09-12T01:33:12.180414+0000","last_unstale":"2024-09-12T01:33:19.413295+0000","last_undegraded":"2024-09-12T01:33:19.413295+0000","last_fullsized":"2024-09-12T01:33:19.413295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:58.901902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.486128+0000","last_change":"2024-09-12T01:33:12.165745+0000","last_active":"2024-09-12T01:33:19.486128+0000","last_peered":"2024-09-12T01:33:19.486128+0000","last_clean":"2024-09-12T01:33:19.486128+0000","last_became_active":"2024-09-12T01:33:12.165475+0000","last_became_peered":"2024-09-12T01:33:12.165475+0000","last_unstale":"2024-09-12T01:33:19.486128+0000","last_undegraded":"2024-09-12T01:33:19.486128+0000","last_fullsized":"2024-09-12T01:33:19.486128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:17.461083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-12T01:33:19.487685+0000","last_change":"2024-09-12T01:33:12.179105+0000","last_active":"2024-09-12T01:33:19.487685+0000","last_peered":"2024-09-12T01:33:19.487685+0000","last_clean":"2024-09-12T01:33:19.487685+0000","last_became_active":"2024-09-12T01:33:12.173779+0000","last_became_peered":"2024-09-12T01:33:12.173779+0000","last_unstale":"2024-09-12T01:33:19.487685+0000","last_undegraded":"2024-09-12T01:33:19.487685+0000","last_fullsized":"2024-09-12T01:33:19.487685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:33:11.121865+0000","last_clean_scrub_stamp":"2024-09-12T01:33:11.121865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:02.950225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":700890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":113784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":110030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509452,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":110066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":282624,"data_stored":113784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":110100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":270336,"data_stored":109940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":876544,"data_stored":700946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:33:37.004 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T01:33:37.004 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T01:33:37.004 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-12T01:33:37.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-12T01:33:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.139+0000 7f841ccba640 1 Processor -- start 2024-09-12T01:33:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.140+0000 7f841ccba640 1 -- start start 2024-09-12T01:33:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.140+0000 7f841ccba640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f841815e760 0x7f84180cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.140+0000 7f841ccba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8418073ec0 con 0x7f841806baf0 2024-09-12T01:33:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.141+0000 7f841ccba640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8418072e20 con 0x7f84180b47f0 2024-09-12T01:33:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.141+0000 7f841ccba640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f8418073100 con 0x7f841815e760 2024-09-12T01:33:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.141+0000 7f841659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f841806baf0 0x7f841806bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46172/0 (socket says 172.21.15.16:46172) 2024-09-12T01:33:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.141+0000 7f841659c640 1 -- 172.21.15.16:0/3205207848 learned_addr learned my addr 172.21.15.16:0/3205207848 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.141+0000 7f8415d9b640 1 --2- 172.21.15.16:0/3205207848 >> v2:172.21.15.33:3300/0 conn(0x7f841815e760 0x7f84180cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.141+0000 7f841559a640 1 -- 172.21.15.16:0/3205207848 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153781568 0 0) 0x7f8418073ec0 con 0x7f841806baf0 2024-09-12T01:33:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.141+0000 7f841559a640 1 -- 172.21.15.16:0/3205207848 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8400003520 con 0x7f841806baf0 2024-09-12T01:33:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.141+0000 7f8415d9b640 1 -- 172.21.15.16:0/3205207848 >> v1:172.21.15.33:6789/0 conn(0x7f84180b47f0 legacy=0x7f84180b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.141+0000 7f8415d9b640 1 -- 172.21.15.16:0/3205207848 >> v1:172.21.15.16:6789/0 conn(0x7f841806baf0 legacy=0x7f841806bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.141+0000 7f841559a640 1 -- 172.21.15.16:0/3205207848 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3385711053 0 0) 0x7f8400003520 con 0x7f841806baf0 2024-09-12T01:33:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.141+0000 7f8415d9b640 1 -- 172.21.15.16:0/3205207848 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84180733e0 con 0x7f841815e760 2024-09-12T01:33:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.142+0000 7f8415d9b640 1 --2- 172.21.15.16:0/3205207848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841815e760 0x7f84180cefe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f840c0097a0 tx=0x7f840c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7d175b0b1c80fd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.142+0000 7f841559a640 1 -- 172.21.15.16:0/3205207848 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c037070 con 0x7f841815e760 2024-09-12T01:33:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.142+0000 7f841559a640 1 -- 172.21.15.16:0/3205207848 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f840c02abc0 con 0x7f841815e760 2024-09-12T01:33:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.142+0000 7f841559a640 1 -- 172.21.15.16:0/3205207848 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408001670 con 0x7f841815e760 2024-09-12T01:33:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.143+0000 7f841ccba640 1 -- 172.21.15.16:0/3205207848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841815e760 msgr2=0x7f84180cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.143+0000 7f841ccba640 1 --2- 172.21.15.16:0/3205207848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841815e760 0x7f84180cefe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f840c0097a0 tx=0x7f840c02a130 comp rx=0 tx=0).stop 2024-09-12T01:33:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.143+0000 7f841ccba640 1 -- 172.21.15.16:0/3205207848 shutdown_connections 2024-09-12T01:33:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.143+0000 7f841ccba640 1 --2- 172.21.15.16:0/3205207848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841815e760 0x7f84180cefe0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.143+0000 7f841ccba640 1 -- 172.21.15.16:0/3205207848 >> 172.21.15.16:0/3205207848 conn(0x7f84180b2c50 msgr2=0x7f84180b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.143+0000 7f841ccba640 1 -- 172.21.15.16:0/3205207848 shutdown_connections 2024-09-12T01:33:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.143+0000 7f841ccba640 1 -- 172.21.15.16:0/3205207848 wait complete. 2024-09-12T01:33:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.144+0000 7f841ccba640 1 Processor -- start 2024-09-12T01:33:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.144+0000 7f841ccba640 1 -- start start 2024-09-12T01:33:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f841ccba640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f841806baf0 0x7f841814fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f841ccba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84180b47f0 0x7f841815abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f841ccba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841815e760 0x7f841815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f841ccba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f841806c850 con 0x7f841815e760 2024-09-12T01:33:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f841ccba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84180739a0 con 0x7f84180b47f0 2024-09-12T01:33:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f841ccba640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f84180cf520 con 0x7f841806baf0 2024-09-12T01:33:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f8416d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841815e760 0x7f841815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f8416d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841815e760 0x7f841815b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60410/0 (socket says 172.21.15.16:60410) 2024-09-12T01:33:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f8416d9d640 1 -- 172.21.15.16:0/335989184 learned_addr learned my addr 172.21.15.16:0/335989184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f8415d9b640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84180b47f0 0x7f841815abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f841659c640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f841806baf0 0x7f841814fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f8416d9d640 1 -- 172.21.15.16:0/335989184 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f841806baf0 msgr2=0x7f841814fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f8416d9d640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f841806baf0 0x7f841814fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f8416d9d640 1 -- 172.21.15.16:0/335989184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84180b47f0 msgr2=0x7f841815abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f8416d9d640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84180b47f0 0x7f841815abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.145+0000 7f8416d9d640 1 -- 172.21.15.16:0/335989184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84181f7ad0 con 0x7f841815e760 2024-09-12T01:33:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.146+0000 7f8416d9d640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841815e760 0x7f841815b0f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f841000c6b0 tx=0x7f841000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.146+0000 7f8406ffd640 1 -- 172.21.15.16:0/335989184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410003b70 con 0x7f841815e760 2024-09-12T01:33:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.146+0000 7f8406ffd640 1 -- 172.21.15.16:0/335989184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8410003d30 con 0x7f841815e760 2024-09-12T01:33:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.146+0000 7f841ccba640 1 -- 172.21.15.16:0/335989184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84181f7db0 con 0x7f841815e760 2024-09-12T01:33:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.146+0000 7f8406ffd640 1 -- 172.21.15.16:0/335989184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410005310 con 0x7f841815e760 2024-09-12T01:33:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.146+0000 7f841ccba640 1 -- 172.21.15.16:0/335989184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8418158150 con 0x7f841815e760 2024-09-12T01:33:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.147+0000 7f841ccba640 1 -- 172.21.15.16:0/335989184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84180cefe0 con 0x7f841815e760 2024-09-12T01:33:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.147+0000 7f8406ffd640 1 -- 172.21.15.16:0/335989184 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84100054b0 con 0x7f841815e760 2024-09-12T01:33:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.148+0000 7f8406ffd640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f83f806be30 0x7f83f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.148+0000 7f841659c640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f83f806be30 0x7f83f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.148+0000 7f8406ffd640 1 -- 172.21.15.16:0/335989184 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f841008ae60 con 0x7f841815e760 2024-09-12T01:33:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.148+0000 7f841659c640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f83f806be30 0x7f83f806e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8408005880 tx=0x7f8408000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.152+0000 7f8406ffd640 1 -- 172.21.15.16:0/335989184 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8410058f10 con 0x7f841815e760 2024-09-12T01:33:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.363+0000 7f841ccba640 1 -- 172.21.15.16:0/335989184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f84180b38b0 con 0x7f841815e760 2024-09-12T01:33:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.364+0000 7f8406ffd640 1 -- 172.21.15.16:0/335989184 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f841005d210 con 0x7f841815e760 2024-09-12T01:33:37.363 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:37.364 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T01:33:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.365+0000 7f841ccba640 1 -- 172.21.15.16:0/335989184 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f83f806be30 msgr2=0x7f83f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.365+0000 7f841ccba640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f83f806be30 0x7f83f806e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8408005880 tx=0x7f8408000bc0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.365+0000 7f841ccba640 1 -- 172.21.15.16:0/335989184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841815e760 msgr2=0x7f841815b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.365+0000 7f841ccba640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841815e760 0x7f841815b0f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f841000c6b0 tx=0x7f841000cb70 comp rx=0 tx=0).stop 2024-09-12T01:33:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.366+0000 7f841ccba640 1 -- 172.21.15.16:0/335989184 shutdown_connections 2024-09-12T01:33:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.366+0000 7f841ccba640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f83f806be30 0x7f83f806e2f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.366+0000 7f841ccba640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841815e760 0x7f841815b0f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.366+0000 7f841ccba640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84180b47f0 0x7f841815abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.366+0000 7f841ccba640 1 --2- 172.21.15.16:0/335989184 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f841806baf0 0x7f841814fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.366+0000 7f841ccba640 1 -- 172.21.15.16:0/335989184 >> 172.21.15.16:0/335989184 conn(0x7f84180b2c50 msgr2=0x7f84180744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.366+0000 7f841ccba640 1 -- 172.21.15.16:0/335989184 shutdown_connections 2024-09-12T01:33:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.366+0000 7f841ccba640 1 -- 172.21.15.16:0/335989184 wait complete. 2024-09-12T01:33:37.376 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-12T01:33:37.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:33:37.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.479+0000 7fb5e4a53640 1 Processor -- start 2024-09-12T01:33:37.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.479+0000 7fb5e4a53640 1 -- start start 2024-09-12T01:33:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5e4a53640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5e015e8d0 0x7fb5e006dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5e4a53640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e006f530 con 0x7fb5e00cb580 2024-09-12T01:33:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5e4a53640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e00cbbb0 con 0x7fb5e00acdd0 2024-09-12T01:33:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5e4a53640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e00cbe90 con 0x7fb5e015e8d0 2024-09-12T01:33:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5ded9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb5e00cb580 0x7fb5e00ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46180/0 (socket says 172.21.15.16:46180) 2024-09-12T01:33:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5ded9d640 1 -- 172.21.15.16:0/1819682308 learned_addr learned my addr 172.21.15.16:0/1819682308 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5de59c640 1 --2- 172.21.15.16:0/1819682308 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5e015e8d0 0x7fb5e006dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1577771815 0 0) 0x7fb5e006f530 con 0x7fb5e00cb580 2024-09-12T01:33:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5c8003520 con 0x7fb5e00cb580 2024-09-12T01:33:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3298623710 0 0) 0x7fb5e00cbbb0 con 0x7fb5e00acdd0 2024-09-12T01:33:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5e006f530 con 0x7fb5e00acdd0 2024-09-12T01:33:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2884030149 0 0) 0x7fb5c8003520 con 0x7fb5e00cb580 2024-09-12T01:33:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5e015e8d0 msgr2=0x7fb5e006dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 --2- 172.21.15.16:0/1819682308 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5e015e8d0 0x7fb5e006dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 >> v1:172.21.15.33:6789/0 conn(0x7fb5e00acdd0 legacy=0x7fb5e00cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e00ca5f0 con 0x7fb5e00cb580 2024-09-12T01:33:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7fb5d80015f0 con 0x7fb5e00cb580 2024-09-12T01:33:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 >> v1:172.21.15.16:6789/0 conn(0x7fb5e00cb580 legacy=0x7fb5e00ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 --2- 172.21.15.16:0/1819682308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5c8003be0 0x7fb5c8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 --2- 172.21.15.16:0/1819682308 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5e015e8d0 0x7fb5c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 --2- 172.21.15.16:0/1819682308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c8004d40 0x7fb5c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e00cbe90 con 0x7fb5c8003be0 2024-09-12T01:33:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c800dd30 con 0x7fb5c8004d40 2024-09-12T01:33:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.480+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c800df30 con 0x7fb5e015e8d0 2024-09-12T01:33:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.481+0000 7fb5de59c640 1 --2- 172.21.15.16:0/1819682308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5c8003be0 0x7fb5c8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.481+0000 7fb5ddd9b640 1 --2- 172.21.15.16:0/1819682308 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5e015e8d0 0x7fb5c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.481+0000 7fb5ded9d640 1 --2- 172.21.15.16:0/1819682308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c8004d40 0x7fb5c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.481+0000 7fb5de59c640 1 -- 172.21.15.16:0/1819682308 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5e015e8d0 msgr2=0x7fb5c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.481+0000 7fb5de59c640 1 --2- 172.21.15.16:0/1819682308 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5e015e8d0 0x7fb5c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.481+0000 7fb5de59c640 1 -- 172.21.15.16:0/1819682308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c8004d40 msgr2=0x7fb5c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.481+0000 7fb5de59c640 1 --2- 172.21.15.16:0/1819682308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c8004d40 0x7fb5c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.481+0000 7fb5de59c640 1 -- 172.21.15.16:0/1819682308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c8003520 con 0x7fb5c8003be0 2024-09-12T01:33:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.482+0000 7fb5de59c640 1 --2- 172.21.15.16:0/1819682308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5c8003be0 0x7fb5c8003fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009fc0 tx=0x7fb5c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a221e86837bd198f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.482+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0037070 con 0x7fb5c8003be0 2024-09-12T01:33:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.482+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d00325f0 con 0x7fb5c8003be0 2024-09-12T01:33:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.482+0000 7fb5dd59a640 1 -- 172.21.15.16:0/1819682308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0009c10 con 0x7fb5c8003be0 2024-09-12T01:33:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.482+0000 7fb5e4a53640 1 -- 172.21.15.16:0/1819682308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5c8003be0 msgr2=0x7fb5c8003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.482+0000 7fb5e4a53640 1 --2- 172.21.15.16:0/1819682308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5c8003be0 0x7fb5c8003fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009fc0 tx=0x7fb5c8005830 comp rx=0 tx=0).stop 2024-09-12T01:33:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.482+0000 7fb5ded9d640 1 -- 172.21.15.16:0/1819682308 reap_dead start 2024-09-12T01:33:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.482+0000 7fb5e4a53640 1 -- 172.21.15.16:0/1819682308 shutdown_connections 2024-09-12T01:33:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.482+0000 7fb5e4a53640 1 -- 172.21.15.16:0/1819682308 >> 172.21.15.16:0/1819682308 conn(0x7fb5e00cffa0 msgr2=0x7fb5e00d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.482+0000 7fb5e4a53640 1 -- 172.21.15.16:0/1819682308 shutdown_connections 2024-09-12T01:33:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.482+0000 7fb5e4a53640 1 -- 172.21.15.16:0/1819682308 wait complete. 2024-09-12T01:33:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.484+0000 7fb5e4a53640 1 Processor -- start 2024-09-12T01:33:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.484+0000 7fb5e4a53640 1 -- start start 2024-09-12T01:33:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.484+0000 7fb5e4a53640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5c8003be0 0x7fb5e01f0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.484+0000 7fb5e4a53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e009d1a0 0x7fb5e00cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.484+0000 7fb5e4a53640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e009d620 0x7fb5e00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.484+0000 7fb5e4a53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e006dd90 con 0x7fb5e009d1a0 2024-09-12T01:33:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.484+0000 7fb5e4a53640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e00cabb0 con 0x7fb5e009d620 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.484+0000 7fb5e4a53640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e006df10 con 0x7fb5c8003be0 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e009d1a0 0x7fb5e00cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e009d1a0 0x7fb5e00cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60428/0 (socket says 172.21.15.16:60428) 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5ddd9b640 1 -- 172.21.15.16:0/633211028 learned_addr learned my addr 172.21.15.16:0/633211028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5de59c640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5c8003be0 0x7fb5e01f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5ded9d640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e009d620 0x7fb5e00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5ddd9b640 1 -- 172.21.15.16:0/633211028 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5c8003be0 msgr2=0x7fb5e01f0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5ddd9b640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5c8003be0 0x7fb5e01f0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5ddd9b640 1 -- 172.21.15.16:0/633211028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e009d620 msgr2=0x7fb5e00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5ddd9b640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e009d620 0x7fb5e00cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5ddd9b640 1 -- 172.21.15.16:0/633211028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e01f0b50 con 0x7fb5e009d1a0 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5ddd9b640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e009d1a0 0x7fb5e00cf050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400f810 tx=0x7fb5d400fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5ceffd640 1 -- 172.21.15.16:0/633211028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d40052e0 con 0x7fb5e009d1a0 2024-09-12T01:33:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5ceffd640 1 -- 172.21.15.16:0/633211028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d4018070 con 0x7fb5e009d1a0 2024-09-12T01:33:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5e4a53640 1 -- 172.21.15.16:0/633211028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e00b4450 con 0x7fb5e009d1a0 2024-09-12T01:33:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5ceffd640 1 -- 172.21.15.16:0/633211028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d400d8f0 con 0x7fb5e009d1a0 2024-09-12T01:33:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.485+0000 7fb5e4a53640 1 -- 172.21.15.16:0/633211028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e00b4cf0 con 0x7fb5e009d1a0 2024-09-12T01:33:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.486+0000 7fb5e4a53640 1 -- 172.21.15.16:0/633211028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e00b47e0 con 0x7fb5e009d1a0 2024-09-12T01:33:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.487+0000 7fb5ceffd640 1 -- 172.21.15.16:0/633211028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5d400da90 con 0x7fb5e009d1a0 2024-09-12T01:33:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.487+0000 7fb5ceffd640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb5c006be60 0x7fb5c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.487+0000 7fb5ceffd640 1 -- 172.21.15.16:0/633211028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb5d408afa0 con 0x7fb5e009d1a0 2024-09-12T01:33:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.487+0000 7fb5de59c640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb5c006be60 0x7fb5c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.488+0000 7fb5de59c640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb5c006be60 0x7fb5c006e320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0034000 tx=0x7fb5d002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.492+0000 7fb5ceffd640 1 -- 172.21.15.16:0/633211028 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb5e01f0b50 con 0x7fb5e009d1a0 2024-09-12T01:33:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.702+0000 7fb5e4a53640 1 -- 172.21.15.16:0/633211028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5e00aff50 con 0x7fb5e009d1a0 2024-09-12T01:33:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.702+0000 7fb5ceffd640 1 -- 172.21.15.16:0/633211028 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7fb5d4059050 con 0x7fb5e009d1a0 2024-09-12T01:33:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:37.702 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-12T01:33:31:251113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4157213575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4157213575},{"type":"v1","addr":"172.21.15.16:6835","nonce":4157213575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1554674078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1554674078},{"type":"v1","addr":"172.21.15.16:6837","nonce":1554674078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/359404074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":359404074},{"type":"v1","addr":"172.21.15.33:6834","nonce":359404074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:33:11.492995+0000","modified":"2024-09-12T01:33:30.603230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2179301157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2179301157},{"type":"v1","addr":"172.21.15.33:6835","nonce":2179301157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-12T01:33:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-12T01:33:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.705+0000 7fb5e4a53640 1 -- 172.21.15.16:0/633211028 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb5c006be60 msgr2=0x7fb5c006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.705+0000 7fb5e4a53640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb5c006be60 0x7fb5c006e320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0034000 tx=0x7fb5d002f000 comp rx=0 tx=0).stop 2024-09-12T01:33:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.705+0000 7fb5e4a53640 1 -- 172.21.15.16:0/633211028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e009d1a0 msgr2=0x7fb5e00cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.705+0000 7fb5e4a53640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e009d1a0 0x7fb5e00cf050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400f810 tx=0x7fb5d400fcd0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.705+0000 7fb5e4a53640 1 -- 172.21.15.16:0/633211028 shutdown_connections 2024-09-12T01:33:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.705+0000 7fb5e4a53640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb5c006be60 0x7fb5c006e320 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.705+0000 7fb5e4a53640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e009d620 0x7fb5e00cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.705+0000 7fb5e4a53640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e009d1a0 0x7fb5e00cf050 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.705+0000 7fb5e4a53640 1 --2- 172.21.15.16:0/633211028 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb5c8003be0 0x7fb5e01f0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.705+0000 7fb5e4a53640 1 -- 172.21.15.16:0/633211028 >> 172.21.15.16:0/633211028 conn(0x7fb5e00cffa0 msgr2=0x7fb5e00b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.705+0000 7fb5e4a53640 1 -- 172.21.15.16:0/633211028 shutdown_connections 2024-09-12T01:33:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.705+0000 7fb5e4a53640 1 -- 172.21.15.16:0/633211028 wait complete. 2024-09-12T01:33:37.716 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:33:37.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.818+0000 7f3c92c65640 1 Processor -- start 2024-09-12T01:33:37.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.818+0000 7f3c92c65640 1 -- start start 2024-09-12T01:33:37.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.819+0000 7f3c92c65640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3c8c15c2b0 0x7f3c8c0af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.819+0000 7f3c92c65640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3c8c0cb930 0x7f3c8c0cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.819+0000 7f3c92c65640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c8c0cb1a0 con 0x7f3c8c0cc540 2024-09-12T01:33:37.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.819+0000 7f3c92c65640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f3c8c071c70 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.819+0000 7f3c92c65640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f3c8c0cb480 con 0x7f3c8c0cb930 2024-09-12T01:33:37.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.820+0000 7f3c90a01640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3c8c15c2b0 0x7f3c8c0af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.820+0000 7f3c91202640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3c8c0cb930 0x7f3c8c0cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.820+0000 7f3c8bfff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3c8c0cc540 0x7f3c8c0738c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:44562/0 (socket says 172.21.15.16:44562) 2024-09-12T01:33:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.820+0000 7f3c8bfff640 1 -- 172.21.15.16:0/917998462 learned_addr learned my addr 172.21.15.16:0/917998462 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:37.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.820+0000 7f3c90a01640 1 -- 172.21.15.16:0/917998462 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3c8c0cb930 msgr2=0x7f3c8c0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:37.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.820+0000 7f3c90a01640 1 --2- 172.21.15.16:0/917998462 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3c8c0cb930 0x7f3c8c0cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.820+0000 7f3c90a01640 1 -- 172.21.15.16:0/917998462 >> v1:172.21.15.33:6789/0 conn(0x7f3c8c0cc540 legacy=0x7f3c8c0738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.820+0000 7f3c8b7fe640 1 -- 172.21.15.16:0/917998462 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1230473468 0 0) 0x7f3c8c0cb1a0 con 0x7f3c8c0cc540 2024-09-12T01:33:37.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.820+0000 7f3c90a01640 1 -- 172.21.15.16:0/917998462 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8c0aec80 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.820+0000 7f3c91202640 1 --2- 172.21.15.16:0/917998462 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3c8c0cb930 0x7f3c8c0cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:37.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.821+0000 7f3c90a01640 1 --2- 172.21.15.16:0/917998462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c8c15c2b0 0x7f3c8c0af060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0097a0 tx=0x7f3c7c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a45de649901bddd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:37.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.821+0000 7f3c8b7fe640 1 -- 172.21.15.16:0/917998462 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c037070 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.821+0000 7f3c8b7fe640 1 -- 172.21.15.16:0/917998462 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c7c032440 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.821+0000 7f3c8b7fe640 1 -- 172.21.15.16:0/917998462 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c032810 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.822+0000 7f3c92c65640 1 -- 172.21.15.16:0/917998462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c8c15c2b0 msgr2=0x7f3c8c0af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.822+0000 7f3c92c65640 1 --2- 172.21.15.16:0/917998462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c8c15c2b0 0x7f3c8c0af060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0097a0 tx=0x7f3c7c02a180 comp rx=0 tx=0).stop 2024-09-12T01:33:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.822+0000 7f3c92c65640 1 -- 172.21.15.16:0/917998462 shutdown_connections 2024-09-12T01:33:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.822+0000 7f3c92c65640 1 --2- 172.21.15.16:0/917998462 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3c8c0cb930 0x7f3c8c0cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.822+0000 7f3c92c65640 1 --2- 172.21.15.16:0/917998462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c8c15c2b0 0x7f3c8c0af060 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.822+0000 7f3c92c65640 1 -- 172.21.15.16:0/917998462 >> 172.21.15.16:0/917998462 conn(0x7f3c8c0cffa0 msgr2=0x7f3c8c0d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.822+0000 7f3c92c65640 1 -- 172.21.15.16:0/917998462 shutdown_connections 2024-09-12T01:33:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.822+0000 7f3c92c65640 1 -- 172.21.15.16:0/917998462 wait complete. 2024-09-12T01:33:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.823+0000 7f3c92c65640 1 Processor -- start 2024-09-12T01:33:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.823+0000 7f3c92c65640 1 -- start start 2024-09-12T01:33:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c92c65640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c8c0cb930 0x7f3c8c14a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c92c65640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3c8c0cc540 0x7f3c8c1f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c92c65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c8c15c2b0 0x7f3c8c1f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c92c65640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c0ad1a0 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c92c65640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c0ad020 con 0x7f3c8c0cb930 2024-09-12T01:33:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c92c65640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f3c8c0ad320 con 0x7f3c8c0cc540 2024-09-12T01:33:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c91202640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c8c15c2b0 0x7f3c8c1f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c91202640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c8c15c2b0 0x7f3c8c1f2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60440/0 (socket says 172.21.15.16:60440) 2024-09-12T01:33:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c91202640 1 -- 172.21.15.16:0/2406090153 learned_addr learned my addr 172.21.15.16:0/2406090153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c90a01640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c8c0cb930 0x7f3c8c14a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c8bfff640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3c8c0cc540 0x7f3c8c1f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c91202640 1 -- 172.21.15.16:0/2406090153 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3c8c0cc540 msgr2=0x7f3c8c1f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c91202640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3c8c0cc540 0x7f3c8c1f1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c91202640 1 -- 172.21.15.16:0/2406090153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c8c0cb930 msgr2=0x7f3c8c14a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c91202640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c8c0cb930 0x7f3c8c14a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c91202640 1 -- 172.21.15.16:0/2406090153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8c0c8f90 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c90a01640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c8c0cb930 0x7f3c8c14a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.824+0000 7f3c91202640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c8c15c2b0 0x7f3c8c1f2470 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400efc0 tx=0x7f3c84009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.825+0000 7f3c897fa640 1 -- 172.21.15.16:0/2406090153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84018030 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.825+0000 7f3c897fa640 1 -- 172.21.15.16:0/2406090153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8400cde0 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.825+0000 7f3c92c65640 1 -- 172.21.15.16:0/2406090153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8c0b2df0 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.825+0000 7f3c897fa640 1 -- 172.21.15.16:0/2406090153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84003bb0 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.825+0000 7f3c92c65640 1 -- 172.21.15.16:0/2406090153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c8c0c94f0 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.826+0000 7f3c92c65640 1 -- 172.21.15.16:0/2406090153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c8c0b2fc0 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.826+0000 7f3c897fa640 1 -- 172.21.15.16:0/2406090153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c84005d10 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.827+0000 7f3c897fa640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f3c7806be30 0x7f3c7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.827+0000 7f3c90a01640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f3c7806be30 0x7f3c7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.827+0000 7f3c897fa640 1 -- 172.21.15.16:0/2406090153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3c8408bbb0 con 0x7f3c8c15c2b0 2024-09-12T01:33:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.827+0000 7f3c90a01640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f3c7806be30 0x7f3c7806e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c034000 tx=0x7f3c7c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:37.831+0000 7f3c897fa640 1 -- 172.21.15.16:0/2406090153 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3c84022070 con 0x7f3c8c15c2b0 2024-09-12T01:33:38.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.041+0000 7f3c92c65640 1 -- 172.21.15.16:0/2406090153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c8c0b5f20 con 0x7f3c8c15c2b0 2024-09-12T01:33:38.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.042+0000 7f3c897fa640 1 -- 172.21.15.16:0/2406090153 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7f3c8c0c8f90 con 0x7f3c8c15c2b0 2024-09-12T01:33:38.042 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-12T01:33:31:251113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4157213575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4157213575},{"type":"v1","addr":"172.21.15.16:6835","nonce":4157213575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1554674078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1554674078},{"type":"v1","addr":"172.21.15.16:6837","nonce":1554674078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/359404074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":359404074},{"type":"v1","addr":"172.21.15.33:6834","nonce":359404074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:33:11.492995+0000","modified":"2024-09-12T01:33:30.603230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2179301157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2179301157},{"type":"v1","addr":"172.21.15.33:6835","nonce":2179301157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-12T01:33:38.042 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-12T01:33:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.044+0000 7f3c92c65640 1 -- 172.21.15.16:0/2406090153 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f3c7806be30 msgr2=0x7f3c7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.044+0000 7f3c92c65640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f3c7806be30 0x7f3c7806e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c034000 tx=0x7f3c7c02e000 comp rx=0 tx=0).stop 2024-09-12T01:33:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.044+0000 7f3c92c65640 1 -- 172.21.15.16:0/2406090153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c8c15c2b0 msgr2=0x7f3c8c1f2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.044+0000 7f3c92c65640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c8c15c2b0 0x7f3c8c1f2470 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400efc0 tx=0x7f3c84009f90 comp rx=0 tx=0).stop 2024-09-12T01:33:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.044+0000 7f3c92c65640 1 -- 172.21.15.16:0/2406090153 shutdown_connections 2024-09-12T01:33:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.044+0000 7f3c92c65640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f3c7806be30 0x7f3c7806e2f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.044+0000 7f3c92c65640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c8c15c2b0 0x7f3c8c1f2470 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.044+0000 7f3c92c65640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f3c8c0cc540 0x7f3c8c1f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.044+0000 7f3c92c65640 1 --2- 172.21.15.16:0/2406090153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c8c0cb930 0x7f3c8c14a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.044+0000 7f3c92c65640 1 -- 172.21.15.16:0/2406090153 >> 172.21.15.16:0/2406090153 conn(0x7f3c8c0cffa0 msgr2=0x7f3c8c0724b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:38.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.044+0000 7f3c92c65640 1 -- 172.21.15.16:0/2406090153 shutdown_connections 2024-09-12T01:33:38.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.044+0000 7f3c92c65640 1 -- 172.21.15.16:0/2406090153 wait complete. 2024-09-12T01:33:38.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:33:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.158+0000 7fa0aad18640 1 Processor -- start 2024-09-12T01:33:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.158+0000 7fa0aad18640 1 -- start start 2024-09-12T01:33:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.159+0000 7fa0aad18640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa0a40d0cc0 0x7fa0a40d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.159+0000 7fa0aad18640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a40b78e0 0x7fa0a40b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.159+0000 7fa0aad18640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a40d3430 con 0x7fa0a415e510 2024-09-12T01:33:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.159+0000 7fa0aad18640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a40d1660 con 0x7fa0a40d0cc0 2024-09-12T01:33:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.159+0000 7fa0aad18640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a40d36d0 con 0x7fa0a40b78e0 2024-09-12T01:33:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.159+0000 7fa0a8ab4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa0a415e510 0x7fa0a40ceb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46192/0 (socket says 172.21.15.16:46192) 2024-09-12T01:33:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.159+0000 7fa0a8ab4640 1 -- 172.21.15.16:0/3948108848 learned_addr learned my addr 172.21.15.16:0/3948108848 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.159+0000 7fa0a3fff640 1 --2- 172.21.15.16:0/3948108848 >> v2:172.21.15.33:3300/0 conn(0x7fa0a40d0cc0 0x7fa0a40d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.159+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3069064843 0 0) 0x7fa0a40d3430 con 0x7fa0a415e510 2024-09-12T01:33:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.159+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa094003520 con 0x7fa0a415e510 2024-09-12T01:33:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.159+0000 7fa0a92b5640 1 --2- 172.21.15.16:0/3948108848 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a40b78e0 0x7fa0a40b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3139645233 0 0) 0x7fa094003520 con 0x7fa0a415e510 2024-09-12T01:33:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a40b78e0 msgr2=0x7fa0a40b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 --2- 172.21.15.16:0/3948108848 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a40b78e0 0x7fa0a40b7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 >> v2:172.21.15.33:3300/0 conn(0x7fa0a40d0cc0 msgr2=0x7fa0a40d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 --2- 172.21.15.16:0/3948108848 >> v2:172.21.15.33:3300/0 conn(0x7fa0a40d0cc0 0x7fa0a40d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a40d39b0 con 0x7fa0a415e510 2024-09-12T01:33:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7fa098001670 con 0x7fa0a415e510 2024-09-12T01:33:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 >> v1:172.21.15.16:6789/0 conn(0x7fa0a415e510 legacy=0x7fa0a40ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 --2- 172.21.15.16:0/3948108848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa094003a20 0x7fa094003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 --2- 172.21.15.16:0/3948108848 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a40b78e0 0x7fa094004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 --2- 172.21.15.16:0/3948108848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa094004d40 0x7fa0940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a40d1660 con 0x7fa094003a20 2024-09-12T01:33:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a40d36d0 con 0x7fa094004d40 2024-09-12T01:33:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa09400ba10 con 0x7fa0a40b78e0 2024-09-12T01:33:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a8ab4640 1 --2- 172.21.15.16:0/3948108848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa094003a20 0x7fa094003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a92b5640 1 --2- 172.21.15.16:0/3948108848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa094004d40 0x7fa0940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a3fff640 1 --2- 172.21.15.16:0/3948108848 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a40b78e0 0x7fa094004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a8ab4640 1 -- 172.21.15.16:0/3948108848 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a40b78e0 msgr2=0x7fa094004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:38.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a8ab4640 1 --2- 172.21.15.16:0/3948108848 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a40b78e0 0x7fa094004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a8ab4640 1 -- 172.21.15.16:0/3948108848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa094004d40 msgr2=0x7fa0940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:38.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a8ab4640 1 --2- 172.21.15.16:0/3948108848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa094004d40 0x7fa0940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a8ab4640 1 -- 172.21.15.16:0/3948108848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa094003520 con 0x7fa094003a20 2024-09-12T01:33:38.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.160+0000 7fa0a92b5640 1 --2- 172.21.15.16:0/3948108848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa094004d40 0x7fa0940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:38.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.161+0000 7fa0a8ab4640 1 --2- 172.21.15.16:0/3948108848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa094003a20 0x7fa094003e40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa098001050 tx=0x7fa094005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8747724cac08a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:38.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.161+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09802e070 con 0x7fa094003a20 2024-09-12T01:33:38.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.161+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa098005af0 con 0x7fa094003a20 2024-09-12T01:33:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.161+0000 7fa0a37fe640 1 -- 172.21.15.16:0/3948108848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098007d70 con 0x7fa094003a20 2024-09-12T01:33:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.161+0000 7fa0aad18640 1 -- 172.21.15.16:0/3948108848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa094003a20 msgr2=0x7fa094003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.161+0000 7fa0aad18640 1 --2- 172.21.15.16:0/3948108848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa094003a20 0x7fa094003e40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa098001050 tx=0x7fa094005830 comp rx=0 tx=0).stop 2024-09-12T01:33:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.161+0000 7fa0a92b5640 1 -- 172.21.15.16:0/3948108848 reap_dead start 2024-09-12T01:33:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.162+0000 7fa0aad18640 1 -- 172.21.15.16:0/3948108848 shutdown_connections 2024-09-12T01:33:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.162+0000 7fa0aad18640 1 -- 172.21.15.16:0/3948108848 >> 172.21.15.16:0/3948108848 conn(0x7fa0a406d1a0 msgr2=0x7fa0a4073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.162+0000 7fa0aad18640 1 -- 172.21.15.16:0/3948108848 shutdown_connections 2024-09-12T01:33:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.162+0000 7fa0aad18640 1 -- 172.21.15.16:0/3948108848 wait complete. 2024-09-12T01:33:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.163+0000 7fa0aad18640 1 Processor -- start 2024-09-12T01:33:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.163+0000 7fa0aad18640 1 -- start start 2024-09-12T01:33:38.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0aad18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa094003a20 0x7fa0a4151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0aad18640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a4151700 0x7fa0a40b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0aad18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0a40b6600 0x7fa0a4148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0aad18640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a40d22a0 con 0x7fa094003a20 2024-09-12T01:33:38.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0aad18640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a40d3f70 con 0x7fa0a40b6600 2024-09-12T01:33:38.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0aad18640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a40cf210 con 0x7fa0a4151700 2024-09-12T01:33:38.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a8ab4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa094003a20 0x7fa0a4151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a8ab4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa094003a20 0x7fa0a4151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60444/0 (socket says 172.21.15.16:60444) 2024-09-12T01:33:38.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a8ab4640 1 -- 172.21.15.16:0/2430264685 learned_addr learned my addr 172.21.15.16:0/2430264685 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:38.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a92b5640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0a40b6600 0x7fa0a4148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a3fff640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a4151700 0x7fa0a40b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a8ab4640 1 -- 172.21.15.16:0/2430264685 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a4151700 msgr2=0x7fa0a40b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a8ab4640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a4151700 0x7fa0a40b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a8ab4640 1 -- 172.21.15.16:0/2430264685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0a40b6600 msgr2=0x7fa0a4148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a8ab4640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0a40b6600 0x7fa0a4148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a8ab4640 1 -- 172.21.15.16:0/2430264685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a40b4290 con 0x7fa094003a20 2024-09-12T01:33:38.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a3fff640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a4151700 0x7fa0a40b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:38.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a8ab4640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa094003a20 0x7fa0a4151be0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c300 tx=0x7fa098000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:38.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a17fa640 1 -- 172.21.15.16:0/2430264685 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098042070 con 0x7fa094003a20 2024-09-12T01:33:38.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a17fa640 1 -- 172.21.15.16:0/2430264685 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa098007480 con 0x7fa094003a20 2024-09-12T01:33:38.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0aad18640 1 -- 172.21.15.16:0/2430264685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a40b4570 con 0x7fa094003a20 2024-09-12T01:33:38.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.164+0000 7fa0a17fa640 1 -- 172.21.15.16:0/2430264685 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098001cc0 con 0x7fa094003a20 2024-09-12T01:33:38.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.165+0000 7fa0aad18640 1 -- 172.21.15.16:0/2430264685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a40b4b30 con 0x7fa094003a20 2024-09-12T01:33:38.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.166+0000 7fa0aad18640 1 -- 172.21.15.16:0/2430264685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a415ab50 con 0x7fa094003a20 2024-09-12T01:33:38.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.166+0000 7fa0a17fa640 1 -- 172.21.15.16:0/2430264685 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa098045070 con 0x7fa094003a20 2024-09-12T01:33:38.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.166+0000 7fa0a17fa640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa08806be30 0x7fa08806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.166+0000 7fa0a17fa640 1 -- 172.21.15.16:0/2430264685 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa0980abb80 con 0x7fa094003a20 2024-09-12T01:33:38.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.166+0000 7fa0a3fff640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa08806be30 0x7fa08806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.167+0000 7fa0a3fff640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa08806be30 0x7fa08806e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4151640 tx=0x7fa090008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:38.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.171+0000 7fa0a17fa640 1 -- 172.21.15.16:0/2430264685 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa098005ac0 con 0x7fa094003a20 2024-09-12T01:33:38.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.350+0000 7fa0aad18640 1 -- 172.21.15.16:0/2430264685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0a415a870 con 0x7fa094003a20 2024-09-12T01:33:38.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.351+0000 7fa0a17fa640 1 -- 172.21.15.16:0/2430264685 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18574 (secure 0 0 0) 0x7fa098079ba0 con 0x7fa094003a20 2024-09-12T01:33:38.351 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:38.351 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:33:18.213622+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:33:09.364639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:33:10.478237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:33:14.204624+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:33:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.353+0000 7fa0aad18640 1 -- 172.21.15.16:0/2430264685 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa08806be30 msgr2=0x7fa08806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.353+0000 7fa0aad18640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa08806be30 0x7fa08806e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4151640 tx=0x7fa090008040 comp rx=0 tx=0).stop 2024-09-12T01:33:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.353+0000 7fa0aad18640 1 -- 172.21.15.16:0/2430264685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa094003a20 msgr2=0x7fa0a4151be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.353+0000 7fa0aad18640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa094003a20 0x7fa0a4151be0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c300 tx=0x7fa098000fe0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.353+0000 7fa0aad18640 1 -- 172.21.15.16:0/2430264685 shutdown_connections 2024-09-12T01:33:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.353+0000 7fa0aad18640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa08806be30 0x7fa08806e2f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.353+0000 7fa0aad18640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0a40b6600 0x7fa0a4148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.353+0000 7fa0aad18640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa0a4151700 0x7fa0a40b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.353+0000 7fa0aad18640 1 --2- 172.21.15.16:0/2430264685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa094003a20 0x7fa0a4151be0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.353+0000 7fa0aad18640 1 -- 172.21.15.16:0/2430264685 >> 172.21.15.16:0/2430264685 conn(0x7fa0a406d1a0 msgr2=0x7fa0a40cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.353+0000 7fa0aad18640 1 -- 172.21.15.16:0/2430264685 shutdown_connections 2024-09-12T01:33:38.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.353+0000 7fa0aad18640 1 -- 172.21.15.16:0/2430264685 wait complete. 2024-09-12T01:33:38.364 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:33:38.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.467+0000 7f1c62b8d640 1 Processor -- start 2024-09-12T01:33:38.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.467+0000 7f1c62b8d640 1 -- start start 2024-09-12T01:33:38.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c62b8d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1c5c0cfae0 0x7f1c5c0d2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c62b8d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1c5c06e6a0 0x7f1c5c0d29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c62b8d640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c5c15ebe0 con 0x7f1c5c15e800 2024-09-12T01:33:38.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c62b8d640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f1c5c0d42d0 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c62b8d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5c0d3ff0 con 0x7f1c5c06e6a0 2024-09-12T01:33:38.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c5bfff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1c5c0cfae0 0x7f1c5c0d2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c6112a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1c5c06e6a0 0x7f1c5c0d29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c60929640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1c5c15e800 0x7f1c5c0cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:44576/0 (socket says 172.21.15.16:44576) 2024-09-12T01:33:38.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c60929640 1 -- 172.21.15.16:0/933845854 learned_addr learned my addr 172.21.15.16:0/933845854 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:38.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c5b7fe640 1 -- 172.21.15.16:0/933845854 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2676660810 0 0) 0x7f1c5c15ebe0 con 0x7f1c5c15e800 2024-09-12T01:33:38.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c6112a640 1 -- 172.21.15.16:0/933845854 >> v2:172.21.15.33:3300/0 conn(0x7f1c5c0cfae0 msgr2=0x7f1c5c0d2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:38.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c6112a640 1 --2- 172.21.15.16:0/933845854 >> v2:172.21.15.33:3300/0 conn(0x7f1c5c0cfae0 0x7f1c5c0d2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c6112a640 1 -- 172.21.15.16:0/933845854 >> v1:172.21.15.33:6789/0 conn(0x7f1c5c15e800 legacy=0x7f1c5c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.468+0000 7f1c6112a640 1 -- 172.21.15.16:0/933845854 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c06cbb0 con 0x7f1c5c06e6a0 2024-09-12T01:33:38.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.469+0000 7f1c6112a640 1 --2- 172.21.15.16:0/933845854 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1c5c06e6a0 0x7f1c5c0d29a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009d90 tx=0x7f1c5402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9ce383b9c656f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:38.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.469+0000 7f1c5b7fe640 1 -- 172.21.15.16:0/933845854 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54007740 con 0x7f1c5c06e6a0 2024-09-12T01:33:38.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.469+0000 7f1c5b7fe640 1 -- 172.21.15.16:0/933845854 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c540078e0 con 0x7f1c5c06e6a0 2024-09-12T01:33:38.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.469+0000 7f1c5b7fe640 1 -- 172.21.15.16:0/933845854 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54007cb0 con 0x7f1c5c06e6a0 2024-09-12T01:33:38.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.470+0000 7f1c62b8d640 1 -- 172.21.15.16:0/933845854 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1c5c06e6a0 msgr2=0x7f1c5c0d29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.470+0000 7f1c62b8d640 1 --2- 172.21.15.16:0/933845854 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1c5c06e6a0 0x7f1c5c0d29a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009d90 tx=0x7f1c5402a660 comp rx=0 tx=0).stop 2024-09-12T01:33:38.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.470+0000 7f1c62b8d640 1 -- 172.21.15.16:0/933845854 shutdown_connections 2024-09-12T01:33:38.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.470+0000 7f1c62b8d640 1 --2- 172.21.15.16:0/933845854 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1c5c06e6a0 0x7f1c5c0d29a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.470+0000 7f1c62b8d640 1 --2- 172.21.15.16:0/933845854 >> v2:172.21.15.33:3300/0 conn(0x7f1c5c0cfae0 0x7f1c5c0d2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.470+0000 7f1c62b8d640 1 -- 172.21.15.16:0/933845854 >> 172.21.15.16:0/933845854 conn(0x7f1c5c09d750 msgr2=0x7f1c5c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:38.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.470+0000 7f1c62b8d640 1 -- 172.21.15.16:0/933845854 shutdown_connections 2024-09-12T01:33:38.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.470+0000 7f1c62b8d640 1 -- 172.21.15.16:0/933845854 wait complete. 2024-09-12T01:33:38.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.471+0000 7f1c62b8d640 1 Processor -- start 2024-09-12T01:33:38.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.471+0000 7f1c62b8d640 1 -- start start 2024-09-12T01:33:38.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.472+0000 7f1c62b8d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1c5c06e6a0 0x7f1c5c1f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.472+0000 7f1c62b8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c0cfae0 0x7f1c5c161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.472+0000 7f1c62b8d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c15e800 0x7f1c5c0b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.472+0000 7f1c62b8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c0d4e30 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.472+0000 7f1c62b8d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c06d170 con 0x7f1c5c15e800 2024-09-12T01:33:38.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.472+0000 7f1c62b8d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5c0d2ee0 con 0x7f1c5c06e6a0 2024-09-12T01:33:38.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.472+0000 7f1c5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c0cfae0 0x7f1c5c161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.472+0000 7f1c5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c0cfae0 0x7f1c5c161120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60452/0 (socket says 172.21.15.16:60452) 2024-09-12T01:33:38.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.472+0000 7f1c5bfff640 1 -- 172.21.15.16:0/3894872733 learned_addr learned my addr 172.21.15.16:0/3894872733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:38.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.472+0000 7f1c6112a640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c15e800 0x7f1c5c0b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.472+0000 7f1c60929640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1c5c06e6a0 0x7f1c5c1f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.473+0000 7f1c5bfff640 1 -- 172.21.15.16:0/3894872733 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1c5c06e6a0 msgr2=0x7f1c5c1f8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.473+0000 7f1c5bfff640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1c5c06e6a0 0x7f1c5c1f8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.473+0000 7f1c5bfff640 1 -- 172.21.15.16:0/3894872733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c15e800 msgr2=0x7f1c5c0b9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.473+0000 7f1c5bfff640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c15e800 0x7f1c5c0b9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.473+0000 7f1c5bfff640 1 -- 172.21.15.16:0/3894872733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c161660 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.473+0000 7f1c5bfff640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c0cfae0 0x7f1c5c161120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800e9b0 tx=0x7f1c4800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:38.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.473+0000 7f1c597fa640 1 -- 172.21.15.16:0/3894872733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48017070 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.473+0000 7f1c597fa640 1 -- 172.21.15.16:0/3894872733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c4800c9a0 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.473+0000 7f1c597fa640 1 -- 172.21.15.16:0/3894872733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4800cd90 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.473+0000 7f1c62b8d640 1 -- 172.21.15.16:0/3894872733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5c0ba060 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.473+0000 7f1c62b8d640 1 -- 172.21.15.16:0/3894872733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5c160640 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.474+0000 7f1c62b8d640 1 -- 172.21.15.16:0/3894872733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c5c068880 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.475+0000 7f1c597fa640 1 -- 172.21.15.16:0/3894872733 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c4801a070 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.475+0000 7f1c597fa640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1c4006be30 0x7f1c4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:38.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.475+0000 7f1c597fa640 1 -- 172.21.15.16:0/3894872733 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1c4808b090 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.475+0000 7f1c60929640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1c4006be30 0x7f1c4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:38.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.476+0000 7f1c60929640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1c4006be30 0x7f1c4006e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1c50007fd0 tx=0x7f1c50007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:38.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.480+0000 7f1c597fa640 1 -- 172.21.15.16:0/3894872733 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1c48059140 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.695+0000 7f1c62b8d640 1 -- 172.21.15.16:0/3894872733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c5c0d07f0 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.695+0000 7f1c597fa640 1 -- 172.21.15.16:0/3894872733 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7f1c4805d440 con 0x7f1c5c0cfae0 2024-09-12T01:33:38.696 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:38.696 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-12T01:33:31:251113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4157213575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4157213575},{"type":"v1","addr":"172.21.15.16:6835","nonce":4157213575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1554674078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1554674078},{"type":"v1","addr":"172.21.15.16:6837","nonce":1554674078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/359404074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":359404074},{"type":"v1","addr":"172.21.15.33:6834","nonce":359404074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:33:11.492995+0000","modified":"2024-09-12T01:33:30.603230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2179301157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2179301157},{"type":"v1","addr":"172.21.15.33:6835","nonce":2179301157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-12T01:33:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-12T01:33:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.697+0000 7f1c62b8d640 1 -- 172.21.15.16:0/3894872733 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1c4006be30 msgr2=0x7f1c4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.697+0000 7f1c62b8d640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1c4006be30 0x7f1c4006e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1c50007fd0 tx=0x7f1c50007de0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.697+0000 7f1c62b8d640 1 -- 172.21.15.16:0/3894872733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c0cfae0 msgr2=0x7f1c5c161120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.697+0000 7f1c62b8d640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c0cfae0 0x7f1c5c161120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800e9b0 tx=0x7f1c4800ee70 comp rx=0 tx=0).stop 2024-09-12T01:33:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.697+0000 7f1c62b8d640 1 -- 172.21.15.16:0/3894872733 shutdown_connections 2024-09-12T01:33:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.697+0000 7f1c62b8d640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1c4006be30 0x7f1c4006e2f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.697+0000 7f1c62b8d640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c15e800 0x7f1c5c0b9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.697+0000 7f1c62b8d640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c0cfae0 0x7f1c5c161120 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.697+0000 7f1c62b8d640 1 --2- 172.21.15.16:0/3894872733 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1c5c06e6a0 0x7f1c5c1f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.698+0000 7f1c62b8d640 1 -- 172.21.15.16:0/3894872733 >> 172.21.15.16:0/3894872733 conn(0x7f1c5c09d750 msgr2=0x7f1c5c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.698+0000 7f1c62b8d640 1 -- 172.21.15.16:0/3894872733 shutdown_connections 2024-09-12T01:33:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:38.698+0000 7f1c62b8d640 1 -- 172.21.15.16:0/3894872733 wait complete. 2024-09-12T01:33:39.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T01:33:11.492995+0000', 'modified': '2024-09-12T01:33:30.603230+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/2179301157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6833', 'nonce': 2179301157}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 2179301157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4287, 'qdb_cluster': [4287]} 2024-09-12T01:33:39.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T01:33:39.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-12T01:33:39.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.219+0000 7f4fdf9f6640 1 Processor -- start 2024-09-12T01:33:39.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.219+0000 7f4fdf9f6640 1 -- start start 2024-09-12T01:33:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.220+0000 7f4fdf9f6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4fd815b350 0x7f4fd815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.220+0000 7f4fdf9f6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4fd815b730 0x7f4fd815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.220+0000 7f4fdf9f6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fd8160000 con 0x7f4fd80cc820 2024-09-12T01:33:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.220+0000 7f4fdf9f6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd815f760 con 0x7f4fd815b730 2024-09-12T01:33:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.220+0000 7f4fdf9f6640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd815fa40 con 0x7f4fd815b350 2024-09-12T01:33:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.221+0000 7f4fdcf91640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4fd815b730 0x7f4fd815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.221+0000 7f4fdcf91640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4fd815b730 0x7f4fd815c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60468/0 (socket says 172.21.15.16:60468) 2024-09-12T01:33:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.221+0000 7f4fdcf91640 1 -- 172.21.15.16:0/574744648 learned_addr learned my addr 172.21.15.16:0/574744648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.221+0000 7f4fdd792640 1 --2- 172.21.15.16:0/574744648 >> v2:172.21.15.33:3300/0 conn(0x7f4fd815b350 0x7f4fd815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.221+0000 7f4fdcf91640 1 -- 172.21.15.16:0/574744648 >> v2:172.21.15.33:3300/0 conn(0x7f4fd815b350 msgr2=0x7f4fd815f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:39.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.221+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/574744648 >> v2:172.21.15.33:3300/0 conn(0x7f4fd815b350 0x7f4fd815f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.221+0000 7f4fdcf91640 1 -- 172.21.15.16:0/574744648 >> v1:172.21.15.16:6789/0 conn(0x7f4fd80cc820 legacy=0x7f4fd80d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.221+0000 7f4fdcf91640 1 -- 172.21.15.16:0/574744648 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd815fd20 con 0x7f4fd815b730 2024-09-12T01:33:39.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.221+0000 7f4fcffff640 1 -- 172.21.15.16:0/574744648 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2027909747 0 0) 0x7f4fd8160000 con 0x7f4fd80cc820 2024-09-12T01:33:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.222+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/574744648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd815b730 0x7f4fd815c5a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8072770 tx=0x7f4fc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3660af32319a372d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.222+0000 7f4fcffff640 1 -- 172.21.15.16:0/574744648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8037070 con 0x7f4fd815b730 2024-09-12T01:33:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.222+0000 7f4fcffff640 1 -- 172.21.15.16:0/574744648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc8032440 con 0x7f4fd815b730 2024-09-12T01:33:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.222+0000 7f4fcffff640 1 -- 172.21.15.16:0/574744648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8032810 con 0x7f4fd815b730 2024-09-12T01:33:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.222+0000 7f4fdf9f6640 1 -- 172.21.15.16:0/574744648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd815b730 msgr2=0x7f4fd815c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.222+0000 7f4fdf9f6640 1 --2- 172.21.15.16:0/574744648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd815b730 0x7f4fd815c5a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8072770 tx=0x7f4fc802a1e0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.222+0000 7f4fdf9f6640 1 -- 172.21.15.16:0/574744648 shutdown_connections 2024-09-12T01:33:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.222+0000 7f4fdf9f6640 1 --2- 172.21.15.16:0/574744648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd815b730 0x7f4fd815c5a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.222+0000 7f4fdf9f6640 1 --2- 172.21.15.16:0/574744648 >> v2:172.21.15.33:3300/0 conn(0x7f4fd815b350 0x7f4fd815f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.222+0000 7f4fdf9f6640 1 -- 172.21.15.16:0/574744648 >> 172.21.15.16:0/574744648 conn(0x7f4fd80b2d10 msgr2=0x7f4fd80b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.223+0000 7f4fdf9f6640 1 -- 172.21.15.16:0/574744648 shutdown_connections 2024-09-12T01:33:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.223+0000 7f4fdf9f6640 1 -- 172.21.15.16:0/574744648 wait complete. 2024-09-12T01:33:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.224+0000 7f4fdf9f6640 1 Processor -- start 2024-09-12T01:33:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.224+0000 7f4fdf9f6640 1 -- start start 2024-09-12T01:33:39.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdf9f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd80cc820 0x7f4fd81587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdf9f6640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4fd815b350 0x7f4fd80cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdf9f6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fd815b730 0x7f4fd80b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdf9f6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd80d08c0 con 0x7f4fd80cc820 2024-09-12T01:33:39.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdf9f6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd80d0740 con 0x7f4fd815b730 2024-09-12T01:33:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdf9f6640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd80ccc30 con 0x7f4fd815b350 2024-09-12T01:33:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdd792640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd80cc820 0x7f4fd81587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdd792640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd80cc820 0x7f4fd81587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60472/0 (socket says 172.21.15.16:60472) 2024-09-12T01:33:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdd792640 1 -- 172.21.15.16:0/3002781350 learned_addr learned my addr 172.21.15.16:0/3002781350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fddf93640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fd815b730 0x7f4fd80b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4fd815b350 0x7f4fd80cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdd792640 1 -- 172.21.15.16:0/3002781350 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4fd815b350 msgr2=0x7f4fd80cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdd792640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4fd815b350 0x7f4fd80cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdd792640 1 -- 172.21.15.16:0/3002781350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fd815b730 msgr2=0x7f4fd80b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdd792640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fd815b730 0x7f4fd80b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdd792640 1 -- 172.21.15.16:0/3002781350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd80c4e80 con 0x7f4fd80cc820 2024-09-12T01:33:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4fd815b350 0x7f4fd80cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdd792640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd80cc820 0x7f4fd81587e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400efc0 tx=0x7f4fd400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fcdffb640 1 -- 172.21.15.16:0/3002781350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4018020 con 0x7f4fd80cc820 2024-09-12T01:33:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fcdffb640 1 -- 172.21.15.16:0/3002781350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd4014480 con 0x7f4fd80cc820 2024-09-12T01:33:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fdf9f6640 1 -- 172.21.15.16:0/3002781350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd80c5160 con 0x7f4fd80cc820 2024-09-12T01:33:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.225+0000 7f4fcdffb640 1 -- 172.21.15.16:0/3002781350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4014850 con 0x7f4fd80cc820 2024-09-12T01:33:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.226+0000 7f4fdf9f6640 1 -- 172.21.15.16:0/3002781350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd80c5720 con 0x7f4fd80cc820 2024-09-12T01:33:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.227+0000 7f4fcdffb640 1 -- 172.21.15.16:0/3002781350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4fd401b020 con 0x7f4fd80cc820 2024-09-12T01:33:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.228+0000 7f4fcdffb640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4fbc06be30 0x7f4fbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.228+0000 7f4fcdffb640 1 -- 172.21.15.16:0/3002781350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4fd408c190 con 0x7f4fd80cc820 2024-09-12T01:33:39.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.228+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4fbc06be30 0x7f4fbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.228+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4fbc06be30 0x7f4fbc06e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8034000 tx=0x7f4fc802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:39.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.228+0000 7f4fe286a740 1 Processor -- start 2024-09-12T01:33:39.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.229+0000 7f4fe286a740 1 -- start start 2024-09-12T01:33:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.229+0000 7f4fe286a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6498500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.229+0000 7f4fe286a740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64b2ae0 0x5616d64b4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.229+0000 7f4fe286a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d646ba80 0x5616d64b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.229+0000 7f4fe286a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5616d63d0520 con 0x5616d646ba80 2024-09-12T01:33:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.229+0000 7f4fe286a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5616d63783e0 con 0x5616d6498120 2024-09-12T01:33:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.229+0000 7f4fe286a740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x5616d63cc4e0 con 0x5616d64b2ae0 2024-09-12T01:33:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.229+0000 7f4fddf93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d646ba80 0x5616d64b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.229+0000 7f4fddf93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d646ba80 0x5616d64b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60484/0 (socket says 172.21.15.16:60484) 2024-09-12T01:33:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.229+0000 7f4fdd792640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6498500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.229+0000 7f4fddf93640 1 -- 172.21.15.16:0/1853137192 learned_addr learned my addr 172.21.15.16:0/1853137192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.229+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/1853137192 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64b2ae0 0x5616d64b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.230+0000 7f4fddf93640 1 -- 172.21.15.16:0/1853137192 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64b2ae0 msgr2=0x5616d64b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.230+0000 7f4fddf93640 1 --2- 172.21.15.16:0/1853137192 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64b2ae0 0x5616d64b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.230+0000 7f4fddf93640 1 -- 172.21.15.16:0/1853137192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 msgr2=0x5616d6498500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.230+0000 7f4fddf93640 1 --2- 172.21.15.16:0/1853137192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6498500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.230+0000 7f4fddf93640 1 -- 172.21.15.16:0/1853137192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616d63932f0 con 0x5616d646ba80 2024-09-12T01:33:39.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.230+0000 7f4fdd792640 1 --2- 172.21.15.16:0/1853137192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6498500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:39.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.230+0000 7f4fddf93640 1 --2- 172.21.15.16:0/1853137192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d646ba80 0x5616d64b47d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00015a0 tx=0x7f4fd0001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed0a16faaeb5ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:39.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.230+0000 7f4fc2ffd640 1 -- 172.21.15.16:0/1853137192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0019070 con 0x5616d646ba80 2024-09-12T01:33:39.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.230+0000 7f4fc2ffd640 1 -- 172.21.15.16:0/1853137192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd00050d0 con 0x5616d646ba80 2024-09-12T01:33:39.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.230+0000 7f4fc2ffd640 1 -- 172.21.15.16:0/1853137192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd000e740 con 0x5616d646ba80 2024-09-12T01:33:39.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.231+0000 7f4fe286a740 1 -- 172.21.15.16:0/1853137192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d646ba80 msgr2=0x5616d64b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.231+0000 7f4fe286a740 1 --2- 172.21.15.16:0/1853137192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d646ba80 0x5616d64b47d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00015a0 tx=0x7f4fd0001a60 comp rx=0 tx=0).stop 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.231+0000 7f4fe286a740 1 -- 172.21.15.16:0/1853137192 shutdown_connections 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.231+0000 7f4fe286a740 1 --2- 172.21.15.16:0/1853137192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d646ba80 0x5616d64b47d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.231+0000 7f4fe286a740 1 --2- 172.21.15.16:0/1853137192 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64b2ae0 0x5616d64b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.231+0000 7f4fe286a740 1 --2- 172.21.15.16:0/1853137192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6498500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.231+0000 7f4fe286a740 1 -- 172.21.15.16:0/1853137192 >> 172.21.15.16:0/1853137192 conn(0x5616d646aa80 msgr2=0x5616d64b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.231+0000 7f4fe286a740 1 -- 172.21.15.16:0/1853137192 shutdown_connections 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.231+0000 7f4fe286a740 1 -- 172.21.15.16:0/1853137192 wait complete. 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.231+0000 7f4fe286a740 1 -- start start 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.231+0000 7f4fe286a740 1 Processor -- start 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.232+0000 7f4fe286a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6494e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.232+0000 7f4fe286a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 0x5616d64abb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.232+0000 7f4fe286a740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64ac070 0x5616d64bdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.232+0000 7f4fe286a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5616d6498ee0 con 0x5616d64b2ae0 2024-09-12T01:33:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.232+0000 7f4fe286a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5616d6359450 con 0x5616d6498120 2024-09-12T01:33:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.232+0000 7f4fe286a740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x5616d64b3770 con 0x5616d64ac070 2024-09-12T01:33:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.232+0000 7f4fe286a740 10 client.0 _ll_register_callbacks cb 0x5616d60e2ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:33:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.232+0000 7f4fdcf91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 0x5616d64abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.232+0000 7f4fdcf91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 0x5616d64abb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60490/0 (socket says 172.21.15.16:60490) 2024-09-12T01:33:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.232+0000 7f4fdcf91640 1 -- 172.21.15.16:0/281279968 learned_addr learned my addr 172.21.15.16:0/281279968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.232+0000 7f4fdd792640 1 --2- 172.21.15.16:0/281279968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6494e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.232+0000 7f4fddf93640 1 --2- 172.21.15.16:0/281279968 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64ac070 0x5616d64bdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fdcf91640 1 -- 172.21.15.16:0/281279968 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64ac070 msgr2=0x5616d64bdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/281279968 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64ac070 0x5616d64bdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fdcf91640 1 -- 172.21.15.16:0/281279968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 msgr2=0x5616d6494e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/281279968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6494e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fdcf91640 1 -- 172.21.15.16:0/281279968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616d60e32d0 con 0x5616d64b2ae0 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/281279968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 0x5616d64abb30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009960 tx=0x7f4fc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4b14508f19d0e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fb7fff640 1 -- 172.21.15.16:0/281279968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8037040 con 0x5616d64b2ae0 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fb7fff640 1 -- 172.21.15.16:0/281279968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4fc8003810 con 0x5616d64b2ae0 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fe286a740 1 -- 172.21.15.16:0/281279968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd00050d0 con 0x5616d64b2ae0 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fb7fff640 1 -- 172.21.15.16:0/281279968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f4fc8055050 con 0x5616d64b2ae0 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fe286a740 10 client.4671 fetch_fsmap learned FSMap version 8 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fe286a740 1 -- 172.21.15.16:0/281279968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5616d61ad6b0 con 0x5616d64b2ae0 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fb7fff640 1 -- 172.21.15.16:0/281279968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd00050d0 con 0x5616d64b2ae0 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.233+0000 7f4fb7fff640 1 -- 172.21.15.16:0/281279968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 8) ==== 3510+0+0 (secure 0 0 0) 0x7f4fc8058020 con 0x5616d64b2ae0 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.234+0000 7f4fe286a740 10 client.4671 fetch_fsmap finished waiting for FSMap version 8 2024-09-12T01:33:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.234+0000 7f4fe286a740 10 client.4671 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.234+0000 7f4fe286a740 20 client.4671 populate_metadata read hostname 'smithi016' 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.234+0000 7f4fe286a740 1 --2- 172.21.15.16:0/281279968 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5616d64cb290 0x5616d64ade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.234+0000 7f4fe286a740 4 client.4671 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.234+0000 7f4fe286a740 1 -- 172.21.15.16:0/281279968 --> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616d63ddc50 con 0x5616d64cb290 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.234+0000 7f4fdd792640 1 --2- 172.21.15.16:0/281279968 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5616d64cb290 0x5616d64ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.234+0000 7f4fb7fff640 10 client.4671 ms_handle_connect on v2:172.21.15.33:6833/2179301157 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.234+0000 7f4fdd792640 1 --2- 172.21.15.16:0/281279968 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5616d64cb290 0x5616d64ade50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eddad4ddbc5710c server_cookie=7f0dd4484da24107 in_seq=0 out_seq=0 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fb7fff640 1 -- 172.21.15.16:0/281279968 <== mds.0 v2:172.21.15.33:6833/2179301157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616d63ddc50 con 0x5616d64cb290 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fb7fff640 10 client.4671 handle_command_reply: tid=0 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fe286a740 1 client.4671 shutdown 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fe286a740 1 -- 172.21.15.16:0/281279968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 msgr2=0x5616d64abb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fe286a740 1 --2- 172.21.15.16:0/281279968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 0x5616d64abb30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009960 tx=0x7f4fc8004730 comp rx=0 tx=0).stop 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fe286a740 1 -- 172.21.15.16:0/281279968 shutdown_connections 2024-09-12T01:33:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fe286a740 1 --2- 172.21.15.16:0/281279968 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5616d64cb290 0x5616d64ade50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fe286a740 1 --2- 172.21.15.16:0/281279968 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64ac070 0x5616d64bdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fe286a740 1 --2- 172.21.15.16:0/281279968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 0x5616d64abb30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fb7fff640 0 client.4671 ms_handle_reset on v2:172.21.15.33:6833/2179301157 2024-09-12T01:33:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fe286a740 1 --2- 172.21.15.16:0/281279968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6494e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fe286a740 1 -- 172.21.15.16:0/281279968 >> 172.21.15.16:0/281279968 conn(0x5616d646ba80 msgr2=0x5616d64b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fe286a740 1 -- 172.21.15.16:0/281279968 shutdown_connections 2024-09-12T01:33:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fe286a740 1 -- 172.21.15.16:0/281279968 wait complete. 2024-09-12T01:33:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.235+0000 7f4fe286a740 20 client.4671 trim_cache size 0 max 16384 2024-09-12T01:33:39.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.253+0000 7f4fe286a740 1 Processor -- start 2024-09-12T01:33:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.253+0000 7f4fe286a740 1 -- start start 2024-09-12T01:33:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fe286a740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d6498120 0x5616d64bc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fe286a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d64ac070 0x5616d646e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fe286a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 0x5616d646e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fe286a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81fc8b0 con 0x5616d64b2ae0 2024-09-12T01:33:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fe286a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd815e520 con 0x5616d64ac070 2024-09-12T01:33:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fe286a740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd80c7920 con 0x5616d6498120 2024-09-12T01:33:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fddf93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 0x5616d646e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fddf93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 0x5616d646e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60502/0 (socket says 172.21.15.16:60502) 2024-09-12T01:33:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fddf93640 1 -- 172.21.15.16:0/4204721715 learned_addr learned my addr 172.21.15.16:0/4204721715 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/4204721715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d64ac070 0x5616d646e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fdd792640 1 --2- 172.21.15.16:0/4204721715 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d6498120 0x5616d64bc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fddf93640 1 -- 172.21.15.16:0/4204721715 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d6498120 msgr2=0x5616d64bc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fddf93640 1 --2- 172.21.15.16:0/4204721715 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d6498120 0x5616d64bc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fddf93640 1 -- 172.21.15.16:0/4204721715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d64ac070 msgr2=0x5616d646e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fddf93640 1 --2- 172.21.15.16:0/4204721715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d64ac070 0x5616d646e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fddf93640 1 -- 172.21.15.16:0/4204721715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616d6498870 con 0x5616d64b2ae0 2024-09-12T01:33:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.254+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/4204721715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d64ac070 0x5616d646e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.255+0000 7f4fddf93640 1 --2- 172.21.15.16:0/4204721715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 0x5616d646e7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0001540 tx=0x7f4fd0005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf61fd4c7730728 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.255+0000 7f4fc27fc640 1 -- 172.21.15.16:0/4204721715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0024070 con 0x5616d64b2ae0 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.255+0000 7f4fc27fc640 1 -- 172.21.15.16:0/4204721715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd000e740 con 0x5616d64b2ae0 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.255+0000 7f4fc27fc640 1 -- 172.21.15.16:0/4204721715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0019040 con 0x5616d64b2ae0 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.255+0000 7f4fe286a740 1 -- 172.21.15.16:0/4204721715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 msgr2=0x5616d646e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.255+0000 7f4fe286a740 1 --2- 172.21.15.16:0/4204721715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 0x5616d646e7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0001540 tx=0x7f4fd0005000 comp rx=0 tx=0).stop 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.255+0000 7f4fe286a740 1 -- 172.21.15.16:0/4204721715 shutdown_connections 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.255+0000 7f4fe286a740 1 --2- 172.21.15.16:0/4204721715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64b2ae0 0x5616d646e7a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.255+0000 7f4fe286a740 1 --2- 172.21.15.16:0/4204721715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d64ac070 0x5616d646e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.255+0000 7f4fe286a740 1 --2- 172.21.15.16:0/4204721715 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d6498120 0x5616d64bc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.255+0000 7f4fe286a740 1 -- 172.21.15.16:0/4204721715 >> 172.21.15.16:0/4204721715 conn(0x5616d646ba80 msgr2=0x5616d64ab870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.255+0000 7f4fe286a740 1 -- 172.21.15.16:0/4204721715 shutdown_connections 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.255+0000 7f4fe286a740 1 -- 172.21.15.16:0/4204721715 wait complete. 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.256+0000 7f4fe286a740 1 -- start start 2024-09-12T01:33:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.256+0000 7f4fe286a740 1 Processor -- start 2024-09-12T01:33:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fe286a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6494c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fe286a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64ac070 0x5616d646e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fe286a740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64b2ae0 0x5616d646e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fe286a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5616d64aee80 con 0x5616d64ac070 2024-09-12T01:33:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fe286a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5616d64ad5b0 con 0x5616d6498120 2024-09-12T01:33:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fe286a740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x5616d64cb6e0 con 0x5616d64b2ae0 2024-09-12T01:33:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fe286a740 10 client.0 _ll_register_callbacks cb 0x7f4fd40095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:33:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fdcf91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64ac070 0x5616d646e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fdcf91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64ac070 0x5616d646e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60512/0 (socket says 172.21.15.16:60512) 2024-09-12T01:33:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fdd792640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6494c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fdcf91640 1 -- 172.21.15.16:0/339699913 learned_addr learned my addr 172.21.15.16:0/339699913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fddf93640 1 --2- 172.21.15.16:0/339699913 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64b2ae0 0x5616d646e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fdcf91640 1 -- 172.21.15.16:0/339699913 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64b2ae0 msgr2=0x5616d646e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/339699913 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64b2ae0 0x5616d646e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fdcf91640 1 -- 172.21.15.16:0/339699913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 msgr2=0x5616d6494c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/339699913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6494c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fdcf91640 1 -- 172.21.15.16:0/339699913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616d64c09f0 con 0x5616d64ac070 2024-09-12T01:33:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.257+0000 7f4fdd792640 1 --2- 172.21.15.16:0/339699913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6494c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fdcf91640 1 --2- 172.21.15.16:0/339699913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64ac070 0x5616d646e240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009930 tx=0x7f4fc803a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d9dff4309863c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:33:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fb7fff640 1 -- 172.21.15.16:0/339699913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8037040 con 0x5616d64ac070 2024-09-12T01:33:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fb7fff640 1 -- 172.21.15.16:0/339699913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4fc8003c70 con 0x5616d64ac070 2024-09-12T01:33:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fe286a740 1 -- 172.21.15.16:0/339699913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd000e740 con 0x5616d64ac070 2024-09-12T01:33:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fb7fff640 1 -- 172.21.15.16:0/339699913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f4fc8069050 con 0x5616d64ac070 2024-09-12T01:33:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fe286a740 10 client.4677 fetch_fsmap learned FSMap version 8 2024-09-12T01:33:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fe286a740 1 -- 172.21.15.16:0/339699913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5616d651f470 con 0x5616d64ac070 2024-09-12T01:33:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fb7fff640 1 -- 172.21.15.16:0/339699913 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd000e740 con 0x5616d64ac070 2024-09-12T01:33:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fb7fff640 1 -- 172.21.15.16:0/339699913 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 8) ==== 3510+0+0 (secure 0 0 0) 0x7f4fc806c020 con 0x5616d64ac070 2024-09-12T01:33:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fe286a740 10 client.4677 fetch_fsmap finished waiting for FSMap version 8 2024-09-12T01:33:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fe286a740 10 client.4677 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T01:33:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fe286a740 20 client.4677 populate_metadata read hostname 'smithi016' 2024-09-12T01:33:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fe286a740 1 --2- 172.21.15.16:0/339699913 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5616d64c9ae0 0x5616d64c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fe286a740 4 client.4677 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:33:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fe286a740 1 -- 172.21.15.16:0/339699913 --> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] -- command(tid 0: {"prefix": "status"}) -- 0x5616d63d7fa0 con 0x5616d64c9ae0 2024-09-12T01:33:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.258+0000 7f4fdd792640 1 --2- 172.21.15.16:0/339699913 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5616d64c9ae0 0x5616d64c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fdd792640 1 --2- 172.21.15.16:0/339699913 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5616d64c9ae0 0x5616d64c9ec0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f78ce822a993da1 server_cookie=79d31d4cbf06a477 in_seq=0 out_seq=0 2024-09-12T01:33:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fb7fff640 10 client.4677 ms_handle_connect on v2:172.21.15.33:6833/2179301157 2024-09-12T01:33:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fb7fff640 1 -- 172.21.15.16:0/339699913 <== mds.0 v2:172.21.15.33:6833/2179301157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x5616d63d7fa0 con 0x5616d64c9ae0 2024-09-12T01:33:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fb7fff640 10 client.4677 handle_command_reply: tid=0 2024-09-12T01:33:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fe286a740 1 client.4677 shutdown 2024-09-12T01:33:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fe286a740 1 -- 172.21.15.16:0/339699913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64ac070 msgr2=0x5616d646e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fe286a740 1 --2- 172.21.15.16:0/339699913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64ac070 0x5616d646e240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009930 tx=0x7f4fc803a000 comp rx=0 tx=0).stop 2024-09-12T01:33:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fe286a740 1 -- 172.21.15.16:0/339699913 shutdown_connections 2024-09-12T01:33:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fe286a740 1 --2- 172.21.15.16:0/339699913 >> [v2:172.21.15.33:6833/2179301157,v1:172.21.15.33:6835/2179301157] conn(0x5616d64c9ae0 0x5616d64c9ec0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fe286a740 1 --2- 172.21.15.16:0/339699913 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x5616d64b2ae0 0x5616d646e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fe286a740 1 --2- 172.21.15.16:0/339699913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616d64ac070 0x5616d646e240 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fe286a740 1 --2- 172.21.15.16:0/339699913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d6498120 0x5616d6494c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fb7fff640 0 client.4677 ms_handle_reset on v2:172.21.15.33:6833/2179301157 2024-09-12T01:33:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fe286a740 1 -- 172.21.15.16:0/339699913 >> 172.21.15.16:0/339699913 conn(0x5616d646ba80 msgr2=0x5616d64677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fe286a740 1 -- 172.21.15.16:0/339699913 shutdown_connections 2024-09-12T01:33:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fe286a740 1 -- 172.21.15.16:0/339699913 wait complete. 2024-09-12T01:33:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.259+0000 7f4fe286a740 20 client.4677 trim_cache size 0 max 16384 2024-09-12T01:33:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:33:39.267 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "cb5903e4-2f7f-485b-9d92-173a6afaa2f8", 2024-09-12T01:33:39.267 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-12T01:33:39.267 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4287, 2024-09-12T01:33:39.267 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T01:33:39.267 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T01:33:39.267 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T01:33:39.267 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 27.11012874, 2024-09-12T01:33:39.267 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 8, 2024-09-12T01:33:39.267 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 26, 2024-09-12T01:33:39.267 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 26, 2024-09-12T01:33:39.267 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 33.133604857999998 2024-09-12T01:33:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:33:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.260+0000 7f4fdf9f6640 1 -- 172.21.15.16:0/3002781350 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4fbc06be30 msgr2=0x7f4fbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.260+0000 7f4fdf9f6640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4fbc06be30 0x7f4fbc06e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8034000 tx=0x7f4fc802e000 comp rx=0 tx=0).stop 2024-09-12T01:33:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.260+0000 7f4fdf9f6640 1 -- 172.21.15.16:0/3002781350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd80cc820 msgr2=0x7f4fd81587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.260+0000 7f4fdf9f6640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd80cc820 0x7f4fd81587e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400efc0 tx=0x7f4fd400c490 comp rx=0 tx=0).stop 2024-09-12T01:33:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.260+0000 7f4fdf9f6640 1 -- 172.21.15.16:0/3002781350 shutdown_connections 2024-09-12T01:33:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.260+0000 7f4fdf9f6640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4fbc06be30 0x7f4fbc06e2f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.260+0000 7f4fdf9f6640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fd815b730 0x7f4fd80b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.260+0000 7f4fdf9f6640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4fd815b350 0x7f4fd80cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.260+0000 7f4fdf9f6640 1 --2- 172.21.15.16:0/3002781350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd80cc820 0x7f4fd81587e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.260+0000 7f4fdf9f6640 1 -- 172.21.15.16:0/3002781350 >> 172.21.15.16:0/3002781350 conn(0x7f4fd80b2d10 msgr2=0x7f4fd815e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.260+0000 7f4fdf9f6640 1 -- 172.21.15.16:0/3002781350 shutdown_connections 2024-09-12T01:33:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.260+0000 7f4fdf9f6640 1 -- 172.21.15.16:0/3002781350 wait complete. 2024-09-12T01:33:39.270 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T01:33:39.280 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T01:33:39.280 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T01:33:39.280 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T01:33:39.281 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-12T01:33:39.297 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-12T01:33:39.298 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-12T01:33:39.298 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-12T01:33:39.298 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T01:33:39.298 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T01:33:39.299 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-12T01:33:39.299 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T01:33:39.299 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-12T01:33:39.299 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-12T01:33:39.299 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T01:33:39.300 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T01:33:39.300 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-12T01:33:39.300 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T01:33:39.300 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-12T01:33:39.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:39.301 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-12T01:33:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-12T01:33:39.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:39.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:33:39.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:39.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-12T01:33:39.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:39.481 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-09-12T01:33:39.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:39.502 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-09-12T01:33:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2024-09-12T01:33:39.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:33:39.578 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-12T01:33:39.578 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-12T01:33:39.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T01:33:39.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.682+0000 7f62ebc3c640 1 Processor -- start 2024-09-12T01:33:39.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.682+0000 7f62ebc3c640 1 -- start start 2024-09-12T01:33:39.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62ebc3c640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f62e415e8d0 0x7f62e406dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62ebc3c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62e40acdd0 0x7f62e40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62ebc3c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e406f530 con 0x7f62e40cb3b0 2024-09-12T01:33:39.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62ebc3c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f62e40cbbb0 con 0x7f62e40acdd0 2024-09-12T01:33:39.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62ebc3c640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f62e40cbe90 con 0x7f62e415e8d0 2024-09-12T01:33:39.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62e91d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62e40acdd0 0x7f62e40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62ea1d9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f62e40cb3b0 0x7f62e40ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46206/0 (socket says 172.21.15.16:46206) 2024-09-12T01:33:39.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62ea1d9640 1 -- 172.21.15.16:0/2032669427 learned_addr learned my addr 172.21.15.16:0/2032669427 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:39.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62e91d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62e40acdd0 0x7f62e40cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60522/0 (socket says 172.21.15.16:60522) 2024-09-12T01:33:39.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62e91d7640 1 -- 172.21.15.16:0/2032669427 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f62e415e8d0 msgr2=0x7f62e406dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:39.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62e91d7640 1 --2- 172.21.15.16:0/2032669427 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f62e415e8d0 0x7f62e406dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62e91d7640 1 -- 172.21.15.16:0/2032669427 >> v1:172.21.15.16:6789/0 conn(0x7f62e40cb3b0 legacy=0x7f62e40ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62e91d7640 1 -- 172.21.15.16:0/2032669427 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e40ca5f0 con 0x7f62e40acdd0 2024-09-12T01:33:39.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.683+0000 7f62e89d6640 1 -- 172.21.15.16:0/2032669427 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2652818387 0 0) 0x7f62e406f530 con 0x7f62e40cb3b0 2024-09-12T01:33:39.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.684+0000 7f62e91d7640 1 --2- 172.21.15.16:0/2032669427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e40acdd0 0x7f62e40cae70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f62d40097d0 tx=0x7f62d402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8014244654649d27 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:39.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.684+0000 7f62e89d6640 1 -- 172.21.15.16:0/2032669427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4037070 con 0x7f62e40acdd0 2024-09-12T01:33:39.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.684+0000 7f62e89d6640 1 -- 172.21.15.16:0/2032669427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d4032440 con 0x7f62e40acdd0 2024-09-12T01:33:39.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.684+0000 7f62e89d6640 1 -- 172.21.15.16:0/2032669427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4032810 con 0x7f62e40acdd0 2024-09-12T01:33:39.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.685+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2032669427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e40acdd0 msgr2=0x7f62e40cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.685+0000 7f62ebc3c640 1 --2- 172.21.15.16:0/2032669427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e40acdd0 0x7f62e40cae70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f62d40097d0 tx=0x7f62d402a1c0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.685+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2032669427 shutdown_connections 2024-09-12T01:33:39.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.685+0000 7f62ebc3c640 1 --2- 172.21.15.16:0/2032669427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e40acdd0 0x7f62e40cae70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.685+0000 7f62ebc3c640 1 --2- 172.21.15.16:0/2032669427 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f62e415e8d0 0x7f62e406dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.685+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2032669427 >> 172.21.15.16:0/2032669427 conn(0x7f62e40cffa0 msgr2=0x7f62e40d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:39.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.685+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2032669427 shutdown_connections 2024-09-12T01:33:39.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.685+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2032669427 wait complete. 2024-09-12T01:33:39.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.686+0000 7f62ebc3c640 1 Processor -- start 2024-09-12T01:33:39.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.686+0000 7f62ebc3c640 1 -- start start 2024-09-12T01:33:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62ebc3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e40acdd0 0x7f62e409c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62ebc3c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62e40cb3b0 0x7f62e409b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62ebc3c640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f62e415e8d0 0x7f62e406de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62ebc3c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62e40cb860 con 0x7f62e40acdd0 2024-09-12T01:33:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62ebc3c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f62e40cabb0 con 0x7f62e40cb3b0 2024-09-12T01:33:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62ebc3c640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f62e4073720 con 0x7f62e415e8d0 2024-09-12T01:33:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62e99d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e40acdd0 0x7f62e409c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62e99d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e40acdd0 0x7f62e409c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60538/0 (socket says 172.21.15.16:60538) 2024-09-12T01:33:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62e99d8640 1 -- 172.21.15.16:0/2083593315 learned_addr learned my addr 172.21.15.16:0/2083593315 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62e91d7640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62e40cb3b0 0x7f62e409b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62ea1d9640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f62e415e8d0 0x7f62e406de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62e99d8640 1 -- 172.21.15.16:0/2083593315 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f62e415e8d0 msgr2=0x7f62e406de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62e99d8640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f62e415e8d0 0x7f62e406de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62e99d8640 1 -- 172.21.15.16:0/2083593315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62e40cb3b0 msgr2=0x7f62e409b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62e99d8640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62e40cb3b0 0x7f62e409b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62e99d8640 1 -- 172.21.15.16:0/2083593315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e409c760 con 0x7f62e40acdd0 2024-09-12T01:33:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.687+0000 7f62e99d8640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e40acdd0 0x7f62e409c060 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62e000b440 tx=0x7f62e000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.688+0000 7f62da7fc640 1 -- 172.21.15.16:0/2083593315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0015020 con 0x7f62e40acdd0 2024-09-12T01:33:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.688+0000 7f62da7fc640 1 -- 172.21.15.16:0/2083593315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e0011890 con 0x7f62e40acdd0 2024-09-12T01:33:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.688+0000 7f62da7fc640 1 -- 172.21.15.16:0/2083593315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0011c80 con 0x7f62e40acdd0 2024-09-12T01:33:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.688+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2083593315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e409ca40 con 0x7f62e40acdd0 2024-09-12T01:33:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.688+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2083593315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e409af60 con 0x7f62e40acdd0 2024-09-12T01:33:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.689+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2083593315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e409cc10 con 0x7f62e40acdd0 2024-09-12T01:33:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.689+0000 7f62da7fc640 1 -- 172.21.15.16:0/2083593315 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62e0018020 con 0x7f62e40acdd0 2024-09-12T01:33:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.690+0000 7f62da7fc640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f62c806be30 0x7f62c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.690+0000 7f62da7fc640 1 -- 172.21.15.16:0/2083593315 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f62e008b250 con 0x7f62e40acdd0 2024-09-12T01:33:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.690+0000 7f62e91d7640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f62c806be30 0x7f62c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:39.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.690+0000 7f62e91d7640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f62c806be30 0x7f62c806e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f62d4034000 tx=0x7f62d402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.694+0000 7f62da7fc640 1 -- 172.21.15.16:0/2083593315 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f62e001f020 con 0x7f62e40acdd0 2024-09-12T01:33:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.906+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2083593315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f62e40ca2b0 con 0x7f62e40acdd0 2024-09-12T01:33:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.906+0000 7f62da7fc640 1 -- 172.21.15.16:0/2083593315 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f62e000d040 con 0x7f62e40acdd0 2024-09-12T01:33:39.906 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T01:33:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.908+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2083593315 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f62c806be30 msgr2=0x7f62c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.908+0000 7f62ebc3c640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f62c806be30 0x7f62c806e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f62d4034000 tx=0x7f62d402e000 comp rx=0 tx=0).stop 2024-09-12T01:33:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.908+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2083593315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e40acdd0 msgr2=0x7f62e409c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.908+0000 7f62ebc3c640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e40acdd0 0x7f62e409c060 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62e000b440 tx=0x7f62e000b900 comp rx=0 tx=0).stop 2024-09-12T01:33:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.908+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2083593315 shutdown_connections 2024-09-12T01:33:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.908+0000 7f62ebc3c640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f62c806be30 0x7f62c806e2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.908+0000 7f62ebc3c640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f62e415e8d0 0x7f62e406de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.908+0000 7f62ebc3c640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62e40cb3b0 0x7f62e409b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.908+0000 7f62ebc3c640 1 --2- 172.21.15.16:0/2083593315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e40acdd0 0x7f62e409c060 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.908+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2083593315 >> 172.21.15.16:0/2083593315 conn(0x7f62e40cffa0 msgr2=0x7f62e4072e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.909+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2083593315 shutdown_connections 2024-09-12T01:33:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:39.909+0000 7f62ebc3c640 1 -- 172.21.15.16:0/2083593315 wait complete. 2024-09-12T01:33:39.919 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T01:33:39.919 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T01:33:39.919 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-12T01:33:39.919 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T01:33:39.919 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T01:33:39.919 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T01:33:39.919 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T01:33:39.919 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T01:33:39.919 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T01:33:39.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:39.920 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-12T01:33:39.935 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T01:33:39.936 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T01:33:39.936 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T01:33:39.936 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:39.936 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-12T01:33:39.936 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:39.936 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:39.936 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:39.936 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-12T01:33:39.936 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:39.936 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:39.936 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-12T01:33:39.937 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T01:33:39.937 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:39.937 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-12T01:33:39.937 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T01:33:39.937 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42742sec preferred_lft 42742sec 2024-09-12T01:33:39.937 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-12T01:33:39.937 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:39.937 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:39.937 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:39.937 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-12T01:33:39.937 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T01:33:39.938 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:39.938 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:39.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-12T01:33:39.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-12T01:33:39.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-12T01:33:39.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T01:33:39.938 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:40.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:40.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:40.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:40.079 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T01:33:40.104 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-12T01:33:40.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:40.105 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-12T01:33:40.162 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-12T01:33:40.162 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T01:33:40.162 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T01:33:40.163 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T01:33:40.163 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T01:33:40.163 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:40.163 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:40.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T01:33:40.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T01:33:40.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T01:33:40.163 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:40.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:40.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:40.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:40.293 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-12T01:33:40.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:40.310 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-12T01:33:40.366 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:40.366 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:40.366 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:33:40.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T01:33:40.367 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:40.485 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T01:33:40.485 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:40.486 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:40.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T01:33:40.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T01:33:40.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T01:33:40.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T01:33:40.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T01:33:40.486 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:40.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:40.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:40.605 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:40.605 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:40.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-12T01:33:40.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T01:33:40.606 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:40.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:40.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:40.685 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-12T01:33:40.685 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:33:40.685 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T01:33:40.701 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T01:33:40.701 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:33:40.701 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T01:33:40.757 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-12T01:33:40.824 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:33:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-12T01:33:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-12T01:33:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-12T01:33:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-12T01:33:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-12T01:33:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-12T01:33:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-12T01:33:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-12T01:33:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-12T01:33:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-12T01:33:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-12T01:33:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-12T01:33:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:33:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-12T01:33:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-12T01:33:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-12T01:33:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-12T01:33:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-12T01:33:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-12T01:33:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:33:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:33:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:33:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:33:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-12T01:33:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-12T01:33:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T01:33:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T01:33:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T01:33:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T01:33:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-12T01:33:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:33:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:33:40.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:40.888 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-12T01:33:40.939 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T01:33:40.940 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-12T01:33:40.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-12T01:33:41.007 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:33:41.066 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-12T01:33:41.066 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-12T01:33:41.066 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-12T01:33:41.066 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-12T01:33:41.066 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-12T01:33:41.066 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-12T01:33:41.066 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-12T01:33:41.067 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-12T01:33:41.067 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-12T01:33:41.067 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-12T01:33:41.067 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-12T01:33:41.067 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-12T01:33:41.067 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:33:41.067 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-12T01:33:41.067 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-12T01:33:41.067 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-12T01:33:41.068 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-12T01:33:41.068 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-12T01:33:41.068 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-12T01:33:41.068 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:33:41.068 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:33:41.068 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:33:41.068 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:33:41.068 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-12T01:33:41.068 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-12T01:33:41.068 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T01:33:41.069 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T01:33:41.069 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T01:33:41.069 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T01:33:41.069 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-12T01:33:41.069 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:33:41.069 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:33:41.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:41.069 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-12T01:33:41.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-12T01:33:41.068+0000 7f9c621d6480 -1 init, newargv = 0x562dde9846f0 newargc=15 2024-09-12T01:33:41.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[44597]: starting ceph client 2024-09-12T01:33:41.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[44597]: starting fuse 2024-09-12T01:33:41.087 INFO:teuthology.orchestra.run.smithi016.stdout:43 2024-09-12T01:33:41.087 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-09-12T01:33:41.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-12T01:33:41.087 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-12T01:33:41.087 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-12T01:33:41.087 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-12T01:33:41.087 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-12T01:33:41.087 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T01:33:41.088 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T01:33:41.089 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-12T01:33:41.089 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-12T01:33:41.089 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-12T01:33:41.089 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T01:33:41.089 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:33:41.089 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-12T01:33:41.089 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-12T01:33:41.207 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.44597.asok 2024-09-12T01:33:41.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:41.218 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44597.asok 2024-09-12T01:33:41.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:41.219 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44597.asok status 2024-09-12T01:33:41.356 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:33:41.356 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-12T01:33:41.356 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T01:33:41.356 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:33:41.357 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-12T01:33:41.357 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-12T01:33:41.357 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T01:33:41.357 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "44597", 2024-09-12T01:33:41.357 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-12T01:33:41.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:33:41.357 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-12T01:33:41.357 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-12T01:33:41.357 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4692, 2024-09-12T01:33:41.358 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-12T01:33:41.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-12T01:33:41.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-12T01:33:41.358 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4692 2024-09-12T01:33:41.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:33:41.358 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-12T01:33:41.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-12T01:33:41.358 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-12T01:33:41.358 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1413386234 2024-09-12T01:33:41.359 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:33:41.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:33:41.359 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-12T01:33:41.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-12T01:33:41.359 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-12T01:33:41.359 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1413386234 2024-09-12T01:33:41.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:33:41.359 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4692 192.168.0.1:0/1413386234", 2024-09-12T01:33:41.359 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1413386234", 2024-09-12T01:33:41.359 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-12T01:33:41.359 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 8, 2024-09-12T01:33:41.360 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 26, 2024-09-12T01:33:41.360 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-12T01:33:41.360 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-12T01:33:41.360 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-12T01:33:41.360 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:33:41.364 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-12T01:33:41.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T01:33:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.467+0000 7f8426f86640 1 Processor -- start 2024-09-12T01:33:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.467+0000 7f8426f86640 1 -- start start 2024-09-12T01:33:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.468+0000 7f8426f86640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f842006dd90 0x7f84200acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.468+0000 7f8426f86640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f84200cb580 0x7f8420073720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.468+0000 7f8426f86640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f842006f530 con 0x7f842015e8d0 2024-09-12T01:33:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.468+0000 7f8426f86640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f84200cbbb0 con 0x7f84200cb580 2024-09-12T01:33:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.468+0000 7f8426f86640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f84200cbe90 con 0x7f842006dd90 2024-09-12T01:33:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.469+0000 7f841ffff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f842006dd90 0x7f84200acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.469+0000 7f8425523640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f84200cb580 0x7f8420073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.469+0000 7f8424d22640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f842015e8d0 0x7f84200cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:44586/0 (socket says 172.21.15.16:44586) 2024-09-12T01:33:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.469+0000 7f8424d22640 1 -- 172.21.15.16:0/3746439835 learned_addr learned my addr 172.21.15.16:0/3746439835 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.469+0000 7f841f7fe640 1 -- 172.21.15.16:0/3746439835 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 699500431 0 0) 0x7f842006f530 con 0x7f842015e8d0 2024-09-12T01:33:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.469+0000 7f841f7fe640 1 -- 172.21.15.16:0/3746439835 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8408005730 con 0x7f842015e8d0 2024-09-12T01:33:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.469+0000 7f8425523640 1 -- 172.21.15.16:0/3746439835 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f842006dd90 msgr2=0x7f84200acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.469+0000 7f8425523640 1 --2- 172.21.15.16:0/3746439835 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f842006dd90 0x7f84200acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.469+0000 7f8425523640 1 -- 172.21.15.16:0/3746439835 >> v1:172.21.15.33:6789/0 conn(0x7f842015e8d0 legacy=0x7f84200cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.469+0000 7f8425523640 1 -- 172.21.15.16:0/3746439835 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84200ca5f0 con 0x7f84200cb580 2024-09-12T01:33:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.469+0000 7f841ffff640 1 --2- 172.21.15.16:0/3746439835 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f842006dd90 0x7f84200acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.470+0000 7f8425523640 1 --2- 172.21.15.16:0/3746439835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84200cb580 0x7f8420073720 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8418009dc0 tx=0x7f841802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8fbce58d3650242 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.470+0000 7f841f7fe640 1 -- 172.21.15.16:0/3746439835 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841802ae60 con 0x7f84200cb580 2024-09-12T01:33:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.470+0000 7f841f7fe640 1 -- 172.21.15.16:0/3746439835 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8418007740 con 0x7f84200cb580 2024-09-12T01:33:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.470+0000 7f841f7fe640 1 -- 172.21.15.16:0/3746439835 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418007af0 con 0x7f84200cb580 2024-09-12T01:33:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.470+0000 7f8426f86640 1 -- 172.21.15.16:0/3746439835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84200cb580 msgr2=0x7f8420073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.470+0000 7f8426f86640 1 --2- 172.21.15.16:0/3746439835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84200cb580 0x7f8420073720 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8418009dc0 tx=0x7f841802a680 comp rx=0 tx=0).stop 2024-09-12T01:33:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.470+0000 7f8426f86640 1 -- 172.21.15.16:0/3746439835 shutdown_connections 2024-09-12T01:33:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.470+0000 7f8426f86640 1 --2- 172.21.15.16:0/3746439835 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f842006dd90 0x7f84200acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.470+0000 7f8426f86640 1 --2- 172.21.15.16:0/3746439835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84200cb580 0x7f8420073720 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.470+0000 7f8426f86640 1 -- 172.21.15.16:0/3746439835 >> 172.21.15.16:0/3746439835 conn(0x7f84200cffa0 msgr2=0x7f84200d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.470+0000 7f8426f86640 1 -- 172.21.15.16:0/3746439835 shutdown_connections 2024-09-12T01:33:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.471+0000 7f8426f86640 1 -- 172.21.15.16:0/3746439835 wait complete. 2024-09-12T01:33:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.472+0000 7f8426f86640 1 Processor -- start 2024-09-12T01:33:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.472+0000 7f8426f86640 1 -- start start 2024-09-12T01:33:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8426f86640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842006dd90 0x7f842009d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8426f86640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84200cb580 0x7f84200c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8426f86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015e8d0 0x7f842015ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8426f86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84200ce860 con 0x7f842015e8d0 2024-09-12T01:33:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8426f86640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84200cabb0 con 0x7f842006dd90 2024-09-12T01:33:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8426f86640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f84200ce9e0 con 0x7f84200cb580 2024-09-12T01:33:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8425523640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015e8d0 0x7f842015ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8425523640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015e8d0 0x7f842015ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60540/0 (socket says 172.21.15.16:60540) 2024-09-12T01:33:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8425523640 1 -- 172.21.15.16:0/211036399 learned_addr learned my addr 172.21.15.16:0/211036399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8424d22640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842006dd90 0x7f842009d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f841ffff640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84200cb580 0x7f84200c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8425523640 1 -- 172.21.15.16:0/211036399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84200cb580 msgr2=0x7f84200c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8425523640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84200cb580 0x7f84200c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8425523640 1 -- 172.21.15.16:0/211036399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842006dd90 msgr2=0x7f842009d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8425523640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842006dd90 0x7f842009d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8425523640 1 -- 172.21.15.16:0/211036399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84200c81f0 con 0x7f842015e8d0 2024-09-12T01:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8425523640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015e8d0 0x7f842015ac00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f841802ac70 tx=0x7f8418004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f841d7fa640 1 -- 172.21.15.16:0/211036399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418040070 con 0x7f842015e8d0 2024-09-12T01:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f841d7fa640 1 -- 172.21.15.16:0/211036399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8418031a40 con 0x7f842015e8d0 2024-09-12T01:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f8426f86640 1 -- 172.21.15.16:0/211036399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84200b43a0 con 0x7f842015e8d0 2024-09-12T01:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.473+0000 7f841d7fa640 1 -- 172.21.15.16:0/211036399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418031c10 con 0x7f842015e8d0 2024-09-12T01:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.474+0000 7f8426f86640 1 -- 172.21.15.16:0/211036399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84200b4960 con 0x7f842015e8d0 2024-09-12T01:33:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.474+0000 7f8426f86640 1 -- 172.21.15.16:0/211036399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84200b4b60 con 0x7f842015e8d0 2024-09-12T01:33:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.475+0000 7f841d7fa640 1 -- 172.21.15.16:0/211036399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8418039040 con 0x7f842015e8d0 2024-09-12T01:33:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.475+0000 7f841d7fa640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f840c06be60 0x7f840c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.476+0000 7f841d7fa640 1 -- 172.21.15.16:0/211036399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f84180ac620 con 0x7f842015e8d0 2024-09-12T01:33:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.476+0000 7f8424d22640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f840c06be60 0x7f840c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.476+0000 7f8424d22640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f840c06be60 0x7f840c06e320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8410005880 tx=0x7f8410009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.481+0000 7f841d7fa640 1 -- 172.21.15.16:0/211036399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f84200b4b60 con 0x7f842015e8d0 2024-09-12T01:33:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.695+0000 7f8426f86640 1 -- 172.21.15.16:0/211036399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f84200adf50 con 0x7f842015e8d0 2024-09-12T01:33:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.695+0000 7f841d7fa640 1 -- 172.21.15.16:0/211036399 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f841807ad20 con 0x7f842015e8d0 2024-09-12T01:33:41.695 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T01:33:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.698+0000 7f8426f86640 1 -- 172.21.15.16:0/211036399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f840c06be60 msgr2=0x7f840c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.698+0000 7f8426f86640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f840c06be60 0x7f840c06e320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8410005880 tx=0x7f8410009040 comp rx=0 tx=0).stop 2024-09-12T01:33:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.698+0000 7f8426f86640 1 -- 172.21.15.16:0/211036399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015e8d0 msgr2=0x7f842015ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.698+0000 7f8426f86640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015e8d0 0x7f842015ac00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f841802ac70 tx=0x7f8418004a40 comp rx=0 tx=0).stop 2024-09-12T01:33:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.698+0000 7f8426f86640 1 -- 172.21.15.16:0/211036399 shutdown_connections 2024-09-12T01:33:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.698+0000 7f8426f86640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f840c06be60 0x7f840c06e320 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.698+0000 7f8426f86640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015e8d0 0x7f842015ac00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.698+0000 7f8426f86640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f84200cb580 0x7f84200c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.698+0000 7f8426f86640 1 --2- 172.21.15.16:0/211036399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842006dd90 0x7f842009d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.698+0000 7f8426f86640 1 -- 172.21.15.16:0/211036399 >> 172.21.15.16:0/211036399 conn(0x7f84200cffa0 msgr2=0x7f84200cd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:41.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.698+0000 7f8426f86640 1 -- 172.21.15.16:0/211036399 shutdown_connections 2024-09-12T01:33:41.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41.698+0000 7f8426f86640 1 -- 172.21.15.16:0/211036399 wait complete. 2024-09-12T01:33:41.708 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T01:33:41.708 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T01:33:41.709 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-12T01:33:41.709 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-12T01:33:41.709 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-12T01:33:41.709 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T01:33:41.709 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T01:33:41.709 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T01:33:41.709 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-12T01:33:41.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:41.709 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-12T01:33:41.726 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T01:33:41.726 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T01:33:41.726 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T01:33:41.726 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:41.726 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-12T01:33:41.726 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:41.726 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:41.726 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-12T01:33:41.727 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-12T01:33:41.727 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T01:33:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-12T01:33:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T01:33:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42741sec preferred_lft 42741sec 2024-09-12T01:33:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-12T01:33:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:41.727 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:41.728 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-12T01:33:41.728 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T01:33:41.728 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether fe:94:45:31:d7:56 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:41.728 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T01:33:41.728 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:41.728 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::2c48:d1ff:feb5:607b/64 scope link 2024-09-12T01:33:41.728 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:41.728 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T01:33:41.728 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether fe:94:45:31:d7:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:33:41.728 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::fc94:45ff:fe31:d756/64 scope link tentative 2024-09-12T01:33:41.728 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:41.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:41.729 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T01:33:41.791 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-12T01:33:41.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:41.792 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-12T01:33:41.849 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-12T01:33:41.850 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T01:33:41.850 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T01:33:41.850 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T01:33:41.850 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T01:33:41.850 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:41.850 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:41.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T01:33:41.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T01:33:41.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T01:33:41.851 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:41.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:41.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:41.978 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-12T01:33:41.993 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T01:33:41.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:41.994 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-12T01:33:42.052 INFO:teuthology.orchestra.run.smithi016.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-12T01:33:42.052 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:42.052 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:42.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T01:33:42.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-12T01:33:42.053 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:42.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:42.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:42.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-12T01:33:42.197 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T01:33:42.198 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T01:33:42.198 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T01:33:42.198 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:42.198 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-12T01:33:42.198 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:42.198 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T01:33:42.198 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 42:09:09:2e:23:dd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-12T01:33:42.198 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-12T01:33:42.198 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:42.198 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::4009:9ff:fe2e:23dd/64 scope link tentative 2024-09-12T01:33:42.198 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:42.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:42.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-12T01:33:42.266 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T01:33:42.266 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T01:33:42.266 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T01:33:42.266 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:42.266 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-12T01:33:42.266 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:42.266 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T01:33:42.266 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 42:09:09:2e:23:dd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-12T01:33:42.267 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-12T01:33:42.267 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:42.267 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::4009:9ff:fe2e:23dd/64 scope link 2024-09-12T01:33:42.267 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:42.268 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-12T01:33:42.268 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:42.268 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:42.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-12T01:33:42.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-12T01:33:42.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-12T01:33:42.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-12T01:33:42.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-12T01:33:42.269 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:42.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:42.424 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:42.424 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:33:42.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.1 up 2024-09-12T01:33:42.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.1 master ceph-brx 2024-09-12T01:33:42.425 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:33:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:42.504 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-12T01:33:42.504 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:33:42.504 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-12T01:33:42.520 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-12T01:33:42.521 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:33:42.521 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-12T01:33:42.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-12T01:33:42.642 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:33:42.698 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-12T01:33:42.698 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-12T01:33:42.698 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-12T01:33:42.699 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-12T01:33:42.700 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:33:42.701 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:33:42.701 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T01:33:42.701 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T01:33:42.701 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T01:33:42.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:42.701 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-12T01:33:42.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:33:42.754 INFO:teuthology.orchestra.run.smithi016.stdout:43 2024-09-12T01:33:42.755 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-09-12T01:33:42.755 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-12T01:33:42.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:33:42.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-12T01:33:42.864 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:33:42.882 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:2024-09-12T01:33:42.882+0000 7f505951e480 -1 init, newargv = 0x55c0448ab710 newargc=15 2024-09-12T01:33:42.882 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:ceph-fuse[45112]: starting ceph client 2024-09-12T01:33:42.886 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:ceph-fuse[45112]: starting fuse 2024-09-12T01:33:42.899 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-12T01:33:42.899 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-12T01:33:42.899 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-12T01:33:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-12T01:33:42.901 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-12T01:33:42.901 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-12T01:33:42.901 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:33:42.901 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:33:42.901 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:33:42.901 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:33:42.901 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-12T01:33:42.901 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-12T01:33:42.901 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T01:33:42.902 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T01:33:42.902 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T01:33:42.902 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T01:33:42.902 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-12T01:33:42.902 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:33:42.902 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:33:42.902 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T01:33:42.902 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T01:33:42.902 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T01:33:42.903 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.2 2024-09-12T01:33:42.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:42.903 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-12T01:33:42.955 INFO:teuthology.orchestra.run.smithi016.stdout:43 2024-09-12T01:33:42.955 INFO:teuthology.orchestra.run.smithi016.stdout:44 2024-09-12T01:33:42.956 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-09-12T01:33:42.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-12T01:33:42.956 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-12T01:33:42.956 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-12T01:33:42.956 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-12T01:33:42.956 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-12T01:33:42.956 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:33:42.956 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-12T01:33:42.956 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-12T01:33:42.956 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-12T01:33:42.956 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-12T01:33:42.957 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T01:33:42.958 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:33:42.958 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.2")) 2024-09-12T01:33:42.958 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-12T01:33:43.075 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.2.45112.asok 2024-09-12T01:33:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:43.087 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.45112.asok 2024-09-12T01:33:43.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:43.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.45112.asok status 2024-09-12T01:33:43.224 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:33:43.224 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-12T01:33:43.224 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T01:33:43.224 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:33:43.224 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "2", 2024-09-12T01:33:43.224 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-12T01:33:43.225 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-12T01:33:43.225 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "45112", 2024-09-12T01:33:43.225 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-12T01:33:43.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:33:43.225 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-12T01:33:43.225 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-12T01:33:43.225 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4704, 2024-09-12T01:33:43.225 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-12T01:33:43.225 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-12T01:33:43.225 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-12T01:33:43.226 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4704 2024-09-12T01:33:43.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:33:43.226 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-12T01:33:43.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-12T01:33:43.226 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.2:0", 2024-09-12T01:33:43.226 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2746536913 2024-09-12T01:33:43.226 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:33:43.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:33:43.226 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-12T01:33:43.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-12T01:33:43.227 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.2:0", 2024-09-12T01:33:43.227 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2746536913 2024-09-12T01:33:43.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:33:43.227 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4704 192.168.0.2:0/2746536913", 2024-09-12T01:33:43.227 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.2:0/2746536913", 2024-09-12T01:33:43.227 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-12T01:33:43.227 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 8, 2024-09-12T01:33:43.227 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 26, 2024-09-12T01:33:43.227 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-12T01:33:43.227 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-12T01:33:43.228 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-12T01:33:43.228 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:33:43.232 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-12T01:33:43.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T01:33:43.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.336+0000 7fe6e469b640 1 Processor -- start 2024-09-12T01:33:43.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.336+0000 7fe6e469b640 1 -- start start 2024-09-12T01:33:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e469b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe6dc15e510 0x7fe6dc0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e469b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6dc0cf040 0x7fe6dc0d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e469b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe6dc0d1200 0x7fe6dc0b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e469b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe6dc0d16b0 con 0x7fe6dc0cf040 2024-09-12T01:33:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e469b640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fe6dc0d3430 con 0x7fe6dc0d1200 2024-09-12T01:33:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e469b640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fe6dc0d36d0 con 0x7fe6dc15e510 2024-09-12T01:33:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e1c36640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6dc0cf040 0x7fe6dc0d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e1c36640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6dc0cf040 0x7fe6dc0d0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60556/0 (socket says 172.21.15.16:60556) 2024-09-12T01:33:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e1c36640 1 -- 172.21.15.16:0/1599310336 learned_addr learned my addr 172.21.15.16:0/1599310336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e2c38640 1 --2- 172.21.15.16:0/1599310336 >> v2:172.21.15.33:3300/0 conn(0x7fe6dc0d1200 0x7fe6dc0b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e2437640 1 --2- 172.21.15.16:0/1599310336 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe6dc15e510 0x7fe6dc0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e1c36640 1 -- 172.21.15.16:0/1599310336 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe6dc15e510 msgr2=0x7fe6dc0ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e1c36640 1 --2- 172.21.15.16:0/1599310336 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe6dc15e510 0x7fe6dc0ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e1c36640 1 -- 172.21.15.16:0/1599310336 >> v2:172.21.15.33:3300/0 conn(0x7fe6dc0d1200 msgr2=0x7fe6dc0b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e1c36640 1 --2- 172.21.15.16:0/1599310336 >> v2:172.21.15.33:3300/0 conn(0x7fe6dc0d1200 0x7fe6dc0b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e1c36640 1 -- 172.21.15.16:0/1599310336 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc0d39b0 con 0x7fe6dc0cf040 2024-09-12T01:33:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.337+0000 7fe6e2437640 1 --2- 172.21.15.16:0/1599310336 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe6dc15e510 0x7fe6dc0ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.338+0000 7fe6e1c36640 1 --2- 172.21.15.16:0/1599310336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0cf040 0x7fe6dc0d0cc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0740c0 tx=0x7fe6cc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa37153763d2bac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.338+0000 7fe6e1435640 1 -- 172.21.15.16:0/1599310336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc037070 con 0x7fe6dc0cf040 2024-09-12T01:33:43.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.338+0000 7fe6e1435640 1 -- 172.21.15.16:0/1599310336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6cc032440 con 0x7fe6dc0cf040 2024-09-12T01:33:43.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.338+0000 7fe6e1435640 1 -- 172.21.15.16:0/1599310336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc032810 con 0x7fe6dc0cf040 2024-09-12T01:33:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.339+0000 7fe6e469b640 1 -- 172.21.15.16:0/1599310336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0cf040 msgr2=0x7fe6dc0d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.339+0000 7fe6e469b640 1 --2- 172.21.15.16:0/1599310336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0cf040 0x7fe6dc0d0cc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0740c0 tx=0x7fe6cc02a140 comp rx=0 tx=0).stop 2024-09-12T01:33:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.339+0000 7fe6e469b640 1 -- 172.21.15.16:0/1599310336 shutdown_connections 2024-09-12T01:33:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.339+0000 7fe6e469b640 1 --2- 172.21.15.16:0/1599310336 >> v2:172.21.15.33:3300/0 conn(0x7fe6dc0d1200 0x7fe6dc0b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.339+0000 7fe6e469b640 1 --2- 172.21.15.16:0/1599310336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0cf040 0x7fe6dc0d0cc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.339+0000 7fe6e469b640 1 --2- 172.21.15.16:0/1599310336 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe6dc15e510 0x7fe6dc0ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.339+0000 7fe6e469b640 1 -- 172.21.15.16:0/1599310336 >> 172.21.15.16:0/1599310336 conn(0x7fe6dc06d1a0 msgr2=0x7fe6dc073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.339+0000 7fe6e469b640 1 -- 172.21.15.16:0/1599310336 shutdown_connections 2024-09-12T01:33:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.339+0000 7fe6e469b640 1 -- 172.21.15.16:0/1599310336 wait complete. 2024-09-12T01:33:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.340+0000 7fe6e469b640 1 Processor -- start 2024-09-12T01:33:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.340+0000 7fe6e469b640 1 -- start start 2024-09-12T01:33:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e469b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0cf040 0x7fe6dc0cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e469b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6dc0d1200 0x7fe6dc15fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e469b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe6dc15e510 0x7fe6dc1520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e469b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc0d22a0 con 0x7fe6dc0cf040 2024-09-12T01:33:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e469b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc0d3f70 con 0x7fe6dc0d1200 2024-09-12T01:33:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e469b640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fe6dc15e8f0 con 0x7fe6dc15e510 2024-09-12T01:33:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e2437640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0cf040 0x7fe6dc0cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e2437640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0cf040 0x7fe6dc0cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60560/0 (socket says 172.21.15.16:60560) 2024-09-12T01:33:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e2437640 1 -- 172.21.15.16:0/979920161 learned_addr learned my addr 172.21.15.16:0/979920161 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e2437640 1 -- 172.21.15.16:0/979920161 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe6dc15e510 msgr2=0x7fe6dc1520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e2437640 1 --2- 172.21.15.16:0/979920161 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe6dc15e510 0x7fe6dc1520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e1c36640 1 --2- 172.21.15.16:0/979920161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6dc0d1200 0x7fe6dc15fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e2437640 1 -- 172.21.15.16:0/979920161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6dc0d1200 msgr2=0x7fe6dc15fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e2437640 1 --2- 172.21.15.16:0/979920161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6dc0d1200 0x7fe6dc15fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e2437640 1 -- 172.21.15.16:0/979920161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc160690 con 0x7fe6dc0cf040 2024-09-12T01:33:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.341+0000 7fe6e2437640 1 --2- 172.21.15.16:0/979920161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0cf040 0x7fe6dc0cdd80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800e760 tx=0x7fe6d800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.342+0000 7fe6d2ffd640 1 -- 172.21.15.16:0/979920161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d800f040 con 0x7fe6dc0cf040 2024-09-12T01:33:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.342+0000 7fe6d2ffd640 1 -- 172.21.15.16:0/979920161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d8004030 con 0x7fe6dc0cf040 2024-09-12T01:33:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.342+0000 7fe6e469b640 1 -- 172.21.15.16:0/979920161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc160970 con 0x7fe6dc0cf040 2024-09-12T01:33:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.342+0000 7fe6d2ffd640 1 -- 172.21.15.16:0/979920161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8004400 con 0x7fe6dc0cf040 2024-09-12T01:33:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.342+0000 7fe6e469b640 1 -- 172.21.15.16:0/979920161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc160f30 con 0x7fe6dc0cf040 2024-09-12T01:33:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.343+0000 7fe6e469b640 1 -- 172.21.15.16:0/979920161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6dc159bc0 con 0x7fe6dc0cf040 2024-09-12T01:33:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.343+0000 7fe6d2ffd640 1 -- 172.21.15.16:0/979920161 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6d800c6d0 con 0x7fe6dc0cf040 2024-09-12T01:33:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.344+0000 7fe6d2ffd640 1 --2- 172.21.15.16:0/979920161 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe6c006be30 0x7fe6c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.344+0000 7fe6d2ffd640 1 -- 172.21.15.16:0/979920161 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe6d808af60 con 0x7fe6dc0cf040 2024-09-12T01:33:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.344+0000 7fe6e1c36640 1 --2- 172.21.15.16:0/979920161 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe6c006be30 0x7fe6c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.344+0000 7fe6e1c36640 1 --2- 172.21.15.16:0/979920161 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe6c006be30 0x7fe6c006e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc034000 tx=0x7fe6cc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.348+0000 7fe6d2ffd640 1 -- 172.21.15.16:0/979920161 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe6dc159bc0 con 0x7fe6dc0cf040 2024-09-12T01:33:43.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.559+0000 7fe6e469b640 1 -- 172.21.15.16:0/979920161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe6dc0cb300 con 0x7fe6dc0cf040 2024-09-12T01:33:43.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.559+0000 7fe6d2ffd640 1 -- 172.21.15.16:0/979920161 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7fe6dc0cb300 con 0x7fe6dc0cf040 2024-09-12T01:33:43.559 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T01:33:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.561+0000 7fe6e469b640 1 -- 172.21.15.16:0/979920161 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe6c006be30 msgr2=0x7fe6c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.561+0000 7fe6e469b640 1 --2- 172.21.15.16:0/979920161 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe6c006be30 0x7fe6c006e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc034000 tx=0x7fe6cc02e000 comp rx=0 tx=0).stop 2024-09-12T01:33:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.561+0000 7fe6e469b640 1 -- 172.21.15.16:0/979920161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0cf040 msgr2=0x7fe6dc0cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.561+0000 7fe6e469b640 1 --2- 172.21.15.16:0/979920161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0cf040 0x7fe6dc0cdd80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800e760 tx=0x7fe6d800ec20 comp rx=0 tx=0).stop 2024-09-12T01:33:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.561+0000 7fe6e469b640 1 -- 172.21.15.16:0/979920161 shutdown_connections 2024-09-12T01:33:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.561+0000 7fe6e469b640 1 --2- 172.21.15.16:0/979920161 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe6c006be30 0x7fe6c006e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.561+0000 7fe6e469b640 1 --2- 172.21.15.16:0/979920161 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe6dc15e510 0x7fe6dc1520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.561+0000 7fe6e469b640 1 --2- 172.21.15.16:0/979920161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6dc0d1200 0x7fe6dc15fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.561+0000 7fe6e469b640 1 --2- 172.21.15.16:0/979920161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0cf040 0x7fe6dc0cdd80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.561+0000 7fe6e469b640 1 -- 172.21.15.16:0/979920161 >> 172.21.15.16:0/979920161 conn(0x7fe6dc06d1a0 msgr2=0x7fe6dc0b78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.561+0000 7fe6e469b640 1 -- 172.21.15.16:0/979920161 shutdown_connections 2024-09-12T01:33:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:43.561+0000 7fe6e469b640 1 -- 172.21.15.16:0/979920161 wait complete. 2024-09-12T01:33:43.572 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T01:33:43.572 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T01:33:43.572 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-09-12T01:33:43.572 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T01:33:43.572 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T01:33:43.572 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T01:33:43.572 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T01:33:43.572 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T01:33:43.572 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T01:33:43.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:43.573 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-09-12T01:33:43.592 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T01:33:43.593 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T01:33:43.593 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T01:33:43.593 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:43.593 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-09-12T01:33:43.593 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:43.593 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:43.593 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:43.593 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-09-12T01:33:43.593 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:43.593 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:43.593 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-09-12T01:33:43.593 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T01:33:43.594 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:43.594 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-09-12T01:33:43.594 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T01:33:43.594 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42736sec preferred_lft 42736sec 2024-09-12T01:33:43.594 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2024-09-12T01:33:43.594 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:43.594 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:43.594 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:43.594 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-09-12T01:33:43.594 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T01:33:43.595 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:43.595 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-12T01:33:43.595 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-09-12T01:33:43.595 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-09-12T01:33:43.595 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-09-12T01:33:43.595 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T01:33:43.595 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-12T01:33:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:43.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:43.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:43.806 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T01:33:43.833 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-09-12T01:33:43.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:43.835 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-09-12T01:33:43.908 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-09-12T01:33:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T01:33:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T01:33:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T01:33:43.909 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T01:33:43.909 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:43.909 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-12T01:33:43.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T01:33:43.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T01:33:43.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T01:33:43.910 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-12T01:33:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:44.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:44.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:44.198 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-09-12T01:33:44.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:44.215 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-09-12T01:33:44.270 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:44.271 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-12T01:33:44.271 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T01:33:44.271 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-12T01:33:44.271 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-12T01:33:44.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:44.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:44.390 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-12T01:33:44.391 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:44.391 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-12T01:33:44.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-12T01:33:44.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T01:33:44.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-12T01:33:44.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-12T01:33:44.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-12T01:33:44.391 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-12T01:33:44.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:44.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:44.543 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:44.543 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-12T01:33:44.543 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-09-12T01:33:44.543 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T01:33:44.543 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-12T01:33:44.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:44.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:44.622 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-12T01:33:44.622 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:33:44.622 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T01:33:44.638 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T01:33:44.638 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:33:44.639 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T01:33:44.694 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-09-12T01:33:44.761 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:33:44.818 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-09-12T01:33:44.818 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-09-12T01:33:44.818 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-09-12T01:33:44.818 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-09-12T01:33:44.819 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-09-12T01:33:44.820 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-09-12T01:33:44.820 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:33:44.820 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:33:44.820 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:33:44.820 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:33:44.820 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-09-12T01:33:44.820 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-09-12T01:33:44.820 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T01:33:44.820 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T01:33:44.820 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T01:33:44.820 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T01:33:44.820 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-09-12T01:33:44.821 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T01:33:44.821 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T01:33:44.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:44.821 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-09-12T01:33:44.874 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T01:33:44.874 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-12T01:33:44.917 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-09-12T01:33:44.942 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:33:45.000 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-09-12T01:33:45.000 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-09-12T01:33:45.000 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-09-12T01:33:45.000 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-09-12T01:33:45.000 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-09-12T01:33:45.001 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-09-12T01:33:45.001 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-09-12T01:33:45.001 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-09-12T01:33:45.001 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-09-12T01:33:45.001 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-09-12T01:33:45.001 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-09-12T01:33:45.001 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-09-12T01:33:45.001 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:33:45.001 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-09-12T01:33:45.001 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-09-12T01:33:45.001 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-09-12T01:33:45.001 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-09-12T01:33:45.001 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T01:33:45.002 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T01:33:45.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:45.003 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-09-12T01:33:45.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi033.stderr:2024-09-12T01:33:45.015+0000 7ff9fa12d480 -1 init, newargv = 0x5576f8f24510 newargc=15 2024-09-12T01:33:45.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi033.stderr:ceph-fuse[43187]: starting ceph client 2024-09-12T01:33:45.019 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi033.stderr:ceph-fuse[43187]: starting fuse 2024-09-12T01:33:45.031 INFO:teuthology.orchestra.run.smithi033.stdout:42 2024-09-12T01:33:45.032 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-12T01:33:45.032 DEBUG:teuthology.orchestra.run.smithi033:> sudo stdin-killer -- python3 -c ' 2024-09-12T01:33:45.032 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-09-12T01:33:45.032 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-09-12T01:33:45.032 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-09-12T01:33:45.032 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-09-12T01:33:45.032 DEBUG:teuthology.orchestra.run.smithi033:> 2024-09-12T01:33:45.032 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-09-12T01:33:45.032 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-12T01:33:45.032 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-09-12T01:33:45.032 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-12T01:33:45.033 DEBUG:teuthology.orchestra.run.smithi033:> 2024-09-12T01:33:45.033 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-09-12T01:33:45.033 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-09-12T01:33:45.033 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-09-12T01:33:45.033 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-09-12T01:33:45.033 DEBUG:teuthology.orchestra.run.smithi033:> 2024-09-12T01:33:45.033 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-09-12T01:33:45.033 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T01:33:45.033 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T01:33:45.034 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T01:33:45.034 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-09-12T01:33:45.034 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-09-12T01:33:45.034 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-09-12T01:33:45.034 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T01:33:45.034 DEBUG:teuthology.orchestra.run.smithi033:> 2024-09-12T01:33:45.034 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.1")) 2024-09-12T01:33:45.034 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-09-12T01:33:45.151 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.1.43187.asok 2024-09-12T01:33:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:45.162 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.43187.asok 2024-09-12T01:33:45.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:45.163 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.43187.asok status 2024-09-12T01:33:45.299 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-12T01:33:45.299 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-09-12T01:33:45.299 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T01:33:45.299 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:33:45.299 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "1", 2024-09-12T01:33:45.300 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-09-12T01:33:45.300 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-12T01:33:45.300 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "43187", 2024-09-12T01:33:45.300 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2024-09-12T01:33:45.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T01:33:45.300 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-09-12T01:33:45.300 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-09-12T01:33:45.300 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4702, 2024-09-12T01:33:45.300 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-09-12T01:33:45.300 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-09-12T01:33:45.300 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-09-12T01:33:45.300 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4702 2024-09-12T01:33:45.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1500512286 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1500512286 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.4702 192.168.0.1:0/1500512286", 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/1500512286", 2024-09-12T01:33:45.301 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2024-09-12T01:33:45.302 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 8, 2024-09-12T01:33:45.302 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 26, 2024-09-12T01:33:45.302 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-09-12T01:33:45.302 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-09-12T01:33:45.302 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-09-12T01:33:45.302 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-12T01:33:45.309 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-12T01:33:45.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T01:33:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.423+0000 7f8ad57ed640 1 Processor -- start 2024-09-12T01:33:45.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.423+0000 7f8ad57ed640 1 -- start start 2024-09-12T01:33:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.424+0000 7f8ad57ed640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ad00cc040 0x7f8ad00af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.424+0000 7f8ad57ed640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ad00cb1a0 con 0x7f8ad015c2b0 2024-09-12T01:33:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.424+0000 7f8ad57ed640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ad0071c70 con 0x7f8ad00cc540 2024-09-12T01:33:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.424+0000 7f8ad57ed640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad00cb480 con 0x7f8ad00cc040 2024-09-12T01:33:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.424+0000 7f8acf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ad00cc040 0x7f8ad00af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.424+0000 7f8aceffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8ad015c2b0 0x7f8ad00738c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46218/0 (socket says 172.21.15.16:46218) 2024-09-12T01:33:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.424+0000 7f8aceffd640 1 -- 172.21.15.16:0/1342451954 learned_addr learned my addr 172.21.15.16:0/1342451954 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.424+0000 7f8acf7fe640 1 -- 172.21.15.16:0/1342451954 >> v1:172.21.15.33:6789/0 conn(0x7f8ad00cc540 legacy=0x7f8ad00cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.424+0000 7f8acf7fe640 1 -- 172.21.15.16:0/1342451954 >> v1:172.21.15.16:6789/0 conn(0x7f8ad015c2b0 legacy=0x7f8ad00738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.424+0000 7f8acdffb640 1 -- 172.21.15.16:0/1342451954 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1029062102 0 0) 0x7f8ad00cb1a0 con 0x7f8ad015c2b0 2024-09-12T01:33:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.424+0000 7f8acf7fe640 1 -- 172.21.15.16:0/1342451954 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad00aec80 con 0x7f8ad00cc040 2024-09-12T01:33:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.425+0000 7f8acf7fe640 1 --2- 172.21.15.16:0/1342451954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad00cc040 0x7f8ad00af060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009d90 tx=0x7f8ac802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3479d74e0465dfac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.425+0000 7f8acdffb640 1 -- 172.21.15.16:0/1342451954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8007740 con 0x7f8ad00cc040 2024-09-12T01:33:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.425+0000 7f8acdffb640 1 -- 172.21.15.16:0/1342451954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac80078e0 con 0x7f8ad00cc040 2024-09-12T01:33:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.425+0000 7f8acdffb640 1 -- 172.21.15.16:0/1342451954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8007cb0 con 0x7f8ad00cc040 2024-09-12T01:33:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.426+0000 7f8ad57ed640 1 -- 172.21.15.16:0/1342451954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad00cc040 msgr2=0x7f8ad00af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.426+0000 7f8ad57ed640 1 --2- 172.21.15.16:0/1342451954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad00cc040 0x7f8ad00af060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009d90 tx=0x7f8ac802a6f0 comp rx=0 tx=0).stop 2024-09-12T01:33:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.426+0000 7f8ad57ed640 1 -- 172.21.15.16:0/1342451954 shutdown_connections 2024-09-12T01:33:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.426+0000 7f8ad57ed640 1 --2- 172.21.15.16:0/1342451954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad00cc040 0x7f8ad00af060 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.426+0000 7f8ad57ed640 1 -- 172.21.15.16:0/1342451954 >> 172.21.15.16:0/1342451954 conn(0x7f8ad00cffa0 msgr2=0x7f8ad00d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:45.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.426+0000 7f8ad57ed640 1 -- 172.21.15.16:0/1342451954 shutdown_connections 2024-09-12T01:33:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.426+0000 7f8ad57ed640 1 -- 172.21.15.16:0/1342451954 wait complete. 2024-09-12T01:33:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.427+0000 7f8ad57ed640 1 Processor -- start 2024-09-12T01:33:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.427+0000 7f8ad57ed640 1 -- start start 2024-09-12T01:33:45.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.428+0000 7f8ad57ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad00cc040 0x7f8ad00b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:45.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.428+0000 7f8ad57ed640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8ad00cc540 0x7f8ad015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:45.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.428+0000 7f8ad57ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad015c2b0 0x7f8ad00d0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:45.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.428+0000 7f8ad57ed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad00ad1a0 con 0x7f8ad00cc040 2024-09-12T01:33:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.428+0000 7f8ad57ed640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad00ad020 con 0x7f8ad015c2b0 2024-09-12T01:33:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.428+0000 7f8ad57ed640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad00ad320 con 0x7f8ad00cc540 2024-09-12T01:33:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.428+0000 7f8aceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad00cc040 0x7f8ad00b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.428+0000 7f8aceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad00cc040 0x7f8ad00b57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60574/0 (socket says 172.21.15.16:60574) 2024-09-12T01:33:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.428+0000 7f8aceffd640 1 -- 172.21.15.16:0/4079020843 learned_addr learned my addr 172.21.15.16:0/4079020843 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.428+0000 7f8acf7fe640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad015c2b0 0x7f8ad00d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.428+0000 7f8ace7fc640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8ad00cc540 0x7f8ad015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.429+0000 7f8aceffd640 1 -- 172.21.15.16:0/4079020843 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8ad00cc540 msgr2=0x7f8ad015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.429+0000 7f8aceffd640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8ad00cc540 0x7f8ad015d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.429+0000 7f8aceffd640 1 -- 172.21.15.16:0/4079020843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad015c2b0 msgr2=0x7f8ad00d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.429+0000 7f8aceffd640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad015c2b0 0x7f8ad00d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.429+0000 7f8aceffd640 1 -- 172.21.15.16:0/4079020843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad01512d0 con 0x7f8ad00cc040 2024-09-12T01:33:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.429+0000 7f8aceffd640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad00cc040 0x7f8ad00b57c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000c420 tx=0x7f8ac000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.429+0000 7f8ab77fe640 1 -- 172.21.15.16:0/4079020843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac00074e0 con 0x7f8ad00cc040 2024-09-12T01:33:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.429+0000 7f8ab77fe640 1 -- 172.21.15.16:0/4079020843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac00047e0 con 0x7f8ad00cc040 2024-09-12T01:33:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.429+0000 7f8ad57ed640 1 -- 172.21.15.16:0/4079020843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad00b5d00 con 0x7f8ad00cc040 2024-09-12T01:33:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.429+0000 7f8ab77fe640 1 -- 172.21.15.16:0/4079020843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0004bd0 con 0x7f8ad00cc040 2024-09-12T01:33:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.429+0000 7f8ad57ed640 1 -- 172.21.15.16:0/4079020843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad0151590 con 0x7f8ad00cc040 2024-09-12T01:33:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.430+0000 7f8ad57ed640 1 -- 172.21.15.16:0/4079020843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad00ad710 con 0x7f8ad00cc040 2024-09-12T01:33:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.431+0000 7f8ab77fe640 1 -- 172.21.15.16:0/4079020843 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ac0011070 con 0x7f8ad00cc040 2024-09-12T01:33:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.431+0000 7f8ab77fe640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8abc06be30 0x7f8abc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.431+0000 7f8ab77fe640 1 -- 172.21.15.16:0/4079020843 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8ac008aba0 con 0x7f8ad00cc040 2024-09-12T01:33:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.431+0000 7f8ace7fc640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8abc06be30 0x7f8abc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.432+0000 7f8ace7fc640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8abc06be30 0x7f8abc06e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ad01f58b0 tx=0x7f8ac4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:45.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.436+0000 7f8ab77fe640 1 -- 172.21.15.16:0/4079020843 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8ac0058c50 con 0x7f8ad00cc040 2024-09-12T01:33:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.652+0000 7f8ad57ed640 1 -- 172.21.15.16:0/4079020843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8ad00b5f20 con 0x7f8ad00cc040 2024-09-12T01:33:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.652+0000 7f8ab77fe640 1 -- 172.21.15.16:0/4079020843 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f8ac008a030 con 0x7f8ad00cc040 2024-09-12T01:33:45.652 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T01:33:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.654+0000 7f8ad57ed640 1 -- 172.21.15.16:0/4079020843 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8abc06be30 msgr2=0x7f8abc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.654+0000 7f8ad57ed640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8abc06be30 0x7f8abc06e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ad01f58b0 tx=0x7f8ac4009040 comp rx=0 tx=0).stop 2024-09-12T01:33:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.654+0000 7f8ad57ed640 1 -- 172.21.15.16:0/4079020843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad00cc040 msgr2=0x7f8ad00b57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.654+0000 7f8ad57ed640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad00cc040 0x7f8ad00b57c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000c420 tx=0x7f8ac000c8e0 comp rx=0 tx=0).stop 2024-09-12T01:33:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.654+0000 7f8ad57ed640 1 -- 172.21.15.16:0/4079020843 shutdown_connections 2024-09-12T01:33:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.654+0000 7f8ad57ed640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f8abc06be30 0x7f8abc06e2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.654+0000 7f8ad57ed640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ad015c2b0 0x7f8ad00d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.654+0000 7f8ad57ed640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f8ad00cc540 0x7f8ad015d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.654+0000 7f8ad57ed640 1 --2- 172.21.15.16:0/4079020843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad00cc040 0x7f8ad00b57c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.654+0000 7f8ad57ed640 1 -- 172.21.15.16:0/4079020843 >> 172.21.15.16:0/4079020843 conn(0x7f8ad00cffa0 msgr2=0x7f8ad00b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.654+0000 7f8ad57ed640 1 -- 172.21.15.16:0/4079020843 shutdown_connections 2024-09-12T01:33:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:45.654+0000 7f8ad57ed640 1 -- 172.21.15.16:0/4079020843 wait complete. 2024-09-12T01:33:45.664 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T01:33:45.665 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T01:33:45.665 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-09-12T01:33:45.665 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-12T01:33:45.665 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-12T01:33:45.665 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T01:33:45.665 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T01:33:45.665 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T01:33:45.665 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-12T01:33:45.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:45.665 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-09-12T01:33:45.683 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T01:33:45.683 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T01:33:45.683 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T01:33:45.683 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:45.683 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-09-12T01:33:45.683 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:45.683 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:45.683 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:45.683 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-09-12T01:33:45.683 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:45.684 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:45.684 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-09-12T01:33:45.684 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T01:33:45.684 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:45.684 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-09-12T01:33:45.684 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T01:33:45.684 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42734sec preferred_lft 42734sec 2024-09-12T01:33:45.684 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2024-09-12T01:33:45.684 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:45.684 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:33:45.684 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:45.684 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-09-12T01:33:45.684 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T01:33:45.685 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 62:4f:98:f2:35:f0 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:33:45.685 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T01:33:45.685 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:45.685 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f406:7dff:feae:99be/64 scope link 2024-09-12T01:33:45.685 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:45.685 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T01:33:45.685 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 62:4f:98:f2:35:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T01:33:45.685 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::604f:98ff:fef2:35f0/64 scope link tentative 2024-09-12T01:33:45.685 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:45.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:45.685 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T01:33:45.749 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-09-12T01:33:45.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:45.750 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-09-12T01:33:45.808 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-09-12T01:33:45.808 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T01:33:45.808 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T01:33:45.808 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T01:33:45.808 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T01:33:45.808 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:45.808 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-12T01:33:45.809 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T01:33:45.809 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T01:33:45.809 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T01:33:45.809 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-12T01:33:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:45.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:45.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:45.940 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-09-12T01:33:45.956 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T01:33:45.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:45.957 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-09-12T01:33:46.012 INFO:teuthology.orchestra.run.smithi033.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-12T01:33:46.012 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:46.012 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-12T01:33:46.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T01:33:46.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-12T01:33:46.013 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-12T01:33:46.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:46.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:46.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:46.132 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-12T01:33:46.157 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T01:33:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T01:33:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T01:33:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-09-12T01:33:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:46.158 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T01:33:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 76:5d:ab:3c:ef:fd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-12T01:33:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-12T01:33:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::745d:abff:fe3c:effd/64 scope link tentative 2024-09-12T01:33:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:46.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:46.159 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-12T01:33:46.224 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T01:33:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T01:33:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T01:33:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-09-12T01:33:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:46.225 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T01:33:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 76:5d:ab:3c:ef:fd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-12T01:33:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-12T01:33:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::745d:abff:fe3c:effd/64 scope link tentative 2024-09-12T01:33:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:33:46.227 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-12T01:33:46.227 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:46.227 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-12T01:33:46.227 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-12T01:33:46.227 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-12T01:33:46.227 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-12T01:33:46.227 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-12T01:33:46.228 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-12T01:33:46.228 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-12T01:33:46.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:46.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:46.384 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:33:46.384 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-12T01:33:46.384 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.1 up 2024-09-12T01:33:46.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.1 master ceph-brx 2024-09-12T01:33:46.385 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-12T01:33:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:33:46.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:46.463 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-12T01:33:46.463 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:33:46.463 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-12T01:33:46.479 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-12T01:33:46.479 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:33:46.479 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-12T01:33:46.535 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-09-12T01:33:46.602 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:33:46.659 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-09-12T01:33:46.659 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-09-12T01:33:46.660 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-09-12T01:33:46.660 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-09-12T01:33:46.660 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-09-12T01:33:46.660 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-09-12T01:33:46.660 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-09-12T01:33:46.660 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-09-12T01:33:46.660 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-09-12T01:33:46.660 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-09-12T01:33:46.660 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-09-12T01:33:46.660 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-09-12T01:33:46.660 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:33:46.660 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-09-12T01:33:46.661 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-09-12T01:33:46.661 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-09-12T01:33:46.661 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-09-12T01:33:46.661 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-09-12T01:33:46.661 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-09-12T01:33:46.661 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:33:46.661 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:33:46.661 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:33:46.661 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:33:46.661 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-09-12T01:33:46.661 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-09-12T01:33:46.661 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T01:33:46.661 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T01:33:46.662 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T01:33:46.662 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T01:33:46.662 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-09-12T01:33:46.662 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T01:33:46.662 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T01:33:46.662 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T01:33:46.662 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T01:33:46.662 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T01:33:46.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:46.662 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-09-12T01:33:46.715 INFO:teuthology.orchestra.run.smithi033.stdout:42 2024-09-12T01:33:46.715 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-09-12T01:33:46.716 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-12T01:33:46.758 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-09-12T01:33:46.784 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:33:46.841 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-09-12T01:33:46.841 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-09-12T01:33:46.841 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-09-12T01:33:46.841 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-09-12T01:33:46.842 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-09-12T01:33:46.842 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-09-12T01:33:46.842 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-09-12T01:33:46.842 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-09-12T01:33:46.842 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-09-12T01:33:46.842 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-09-12T01:33:46.842 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-09-12T01:33:46.842 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-09-12T01:33:46.842 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:33:46.842 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-09-12T01:33:46.842 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-09-12T01:33:46.842 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-09-12T01:33:46.843 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-09-12T01:33:46.843 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-09-12T01:33:46.843 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-09-12T01:33:46.843 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:33:46.843 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:33:46.843 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:33:46.843 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:33:46.843 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-09-12T01:33:46.843 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-09-12T01:33:46.843 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T01:33:46.844 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T01:33:46.844 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T01:33:46.844 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T01:33:46.844 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-09-12T01:33:46.844 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T01:33:46.844 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T01:33:46.844 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T01:33:46.844 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T01:33:46.844 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T01:33:46.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:46.845 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-09-12T01:33:46.846 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi033.stderr:2024-09-12T01:33:46.843+0000 7f42fab11480 -1 init, newargv = 0x5613aefcf320 newargc=15 2024-09-12T01:33:46.846 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi033.stderr:ceph-fuse[43636]: starting ceph client 2024-09-12T01:33:46.849 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi033.stderr:ceph-fuse[43636]: starting fuse 2024-09-12T01:33:46.862 INFO:teuthology.orchestra.run.smithi033.stdout:42 2024-09-12T01:33:46.862 INFO:teuthology.orchestra.run.smithi033.stdout:44 2024-09-12T01:33:46.862 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 44] 2024-09-12T01:33:46.862 DEBUG:teuthology.orchestra.run.smithi033:> sudo stdin-killer -- python3 -c ' 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-09-12T01:33:46.863 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-09-12T01:33:46.864 DEBUG:teuthology.orchestra.run.smithi033:> 2024-09-12T01:33:46.864 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-09-12T01:33:46.864 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T01:33:46.864 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T01:33:46.864 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T01:33:46.864 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-09-12T01:33:46.864 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-09-12T01:33:46.864 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-09-12T01:33:46.864 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T01:33:46.864 DEBUG:teuthology.orchestra.run.smithi033:> 2024-09-12T01:33:46.864 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.3")) 2024-09-12T01:33:46.864 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-09-12T01:33:46.982 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.3.43636.asok 2024-09-12T01:33:46.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-12T01:33:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:33:46.993 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.43636.asok 2024-09-12T01:33:46.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:46.993 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.43636.asok status 2024-09-12T01:33:47.130 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-12T01:33:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-09-12T01:33:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T01:33:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:33:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "3", 2024-09-12T01:33:47.130 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-09-12T01:33:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-12T01:33:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "43636", 2024-09-12T01:33:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2024-09-12T01:33:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T01:33:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-09-12T01:33:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-09-12T01:33:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4553, 2024-09-12T01:33:47.131 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4553 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.2:0", 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 4027196791 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.2:0", 2024-09-12T01:33:47.132 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 4027196791 2024-09-12T01:33:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-12T01:33:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.4553 192.168.0.2:0/4027196791", 2024-09-12T01:33:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.2:0/4027196791", 2024-09-12T01:33:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2024-09-12T01:33:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 8, 2024-09-12T01:33:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 26, 2024-09-12T01:33:47.133 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-09-12T01:33:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-09-12T01:33:47.134 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-09-12T01:33:47.134 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-12T01:33:47.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:47.139 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-12T01:33:47.139 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T01:33:47.160 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-12T01:33:47.160 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T01:33:47.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:47.160 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T01:33:47.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:47.232 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-12T01:33:47.233 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-12T01:33:47.292 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-12T01:33:47.293 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-12T01:33:47.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:47.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-12T01:33:47.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:47.363 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-09-12T01:33:47.363 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-12T01:33:47.380 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-09-12T01:33:47.380 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-12T01:33:47.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:47.380 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T01:33:47.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:47.447 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-09-12T01:33:47.447 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-12T01:33:47.504 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-09-12T01:33:47.504 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-12T01:33:47.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:47.504 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-12T01:33:47.571 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-12T01:33:47.614 INFO:tasks.cephfs_test_runner:Starting test: test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) MDS starts to handle client caps when it enters clientreplay stage. 2024-09-12T01:33:47.614 INFO:tasks.cephfs_test_runner:test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) 2024-09-12T01:33:47.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:33:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.729+0000 7f5acc8af640 1 Processor -- start 2024-09-12T01:33:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.730+0000 7f5acc8af640 1 -- start start 2024-09-12T01:33:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.730+0000 7f5acc8af640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5ac815ff00 0x7f5ac80cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.730+0000 7f5acc8af640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ac8160360 0x7f5ac80ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.730+0000 7f5acc8af640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac81608e0 con 0x7f5ac8156940 2024-09-12T01:33:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.730+0000 7f5acc8af640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac815ce90 con 0x7f5ac8160360 2024-09-12T01:33:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.730+0000 7f5acc8af640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac8160bc0 con 0x7f5ac815ff00 2024-09-12T01:33:47.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.731+0000 7f5ac6d9d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ac8160360 0x7f5ac80ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:47.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.731+0000 7f5ac5d9b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5ac815ff00 0x7f5ac80cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:47.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.731+0000 7f5ac659c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5ac8156940 0x7f5ac815d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:49780/0 (socket says 172.21.15.16:49780) 2024-09-12T01:33:47.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.731+0000 7f5ac659c640 1 -- 172.21.15.16:0/3942390261 learned_addr learned my addr 172.21.15.16:0/3942390261 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:47.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.731+0000 7f5ac559a640 1 -- 172.21.15.16:0/3942390261 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 107758505 0 0) 0x7f5ac81608e0 con 0x7f5ac8156940 2024-09-12T01:33:47.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.731+0000 7f5ac559a640 1 -- 172.21.15.16:0/3942390261 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ab0003520 con 0x7f5ac8156940 2024-09-12T01:33:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.731+0000 7f5ac6d9d640 1 -- 172.21.15.16:0/3942390261 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5ac815ff00 msgr2=0x7f5ac80cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.731+0000 7f5ac6d9d640 1 --2- 172.21.15.16:0/3942390261 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5ac815ff00 0x7f5ac80cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.731+0000 7f5ac6d9d640 1 -- 172.21.15.16:0/3942390261 >> v1:172.21.15.33:6789/0 conn(0x7f5ac8156940 legacy=0x7f5ac815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.731+0000 7f5ac6d9d640 1 -- 172.21.15.16:0/3942390261 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac815cb20 con 0x7f5ac8160360 2024-09-12T01:33:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.731+0000 7f5ac5d9b640 1 --2- 172.21.15.16:0/3942390261 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5ac815ff00 0x7f5ac80cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:47.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.732+0000 7f5ac6d9d640 1 --2- 172.21.15.16:0/3942390261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8160360 0x7f5ac80ccd60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009d90 tx=0x7f5ac002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91fb657cd79cae6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:47.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.732+0000 7f5ac559a640 1 -- 172.21.15.16:0/3942390261 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0007740 con 0x7f5ac8160360 2024-09-12T01:33:47.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.732+0000 7f5ac559a640 1 -- 172.21.15.16:0/3942390261 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac00078e0 con 0x7f5ac8160360 2024-09-12T01:33:47.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.732+0000 7f5ac559a640 1 -- 172.21.15.16:0/3942390261 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0007cb0 con 0x7f5ac8160360 2024-09-12T01:33:47.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.733+0000 7f5acc8af640 1 -- 172.21.15.16:0/3942390261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8160360 msgr2=0x7f5ac80ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:47.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.733+0000 7f5acc8af640 1 --2- 172.21.15.16:0/3942390261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8160360 0x7f5ac80ccd60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009d90 tx=0x7f5ac002a660 comp rx=0 tx=0).stop 2024-09-12T01:33:47.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.733+0000 7f5acc8af640 1 -- 172.21.15.16:0/3942390261 shutdown_connections 2024-09-12T01:33:47.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.733+0000 7f5acc8af640 1 --2- 172.21.15.16:0/3942390261 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5ac815ff00 0x7f5ac80cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:47.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.733+0000 7f5acc8af640 1 --2- 172.21.15.16:0/3942390261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8160360 0x7f5ac80ccd60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:47.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.733+0000 7f5acc8af640 1 -- 172.21.15.16:0/3942390261 >> 172.21.15.16:0/3942390261 conn(0x7f5ac806fdc0 msgr2=0x7f5ac80701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:47.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.733+0000 7f5acc8af640 1 -- 172.21.15.16:0/3942390261 shutdown_connections 2024-09-12T01:33:47.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.733+0000 7f5acc8af640 1 -- 172.21.15.16:0/3942390261 wait complete. 2024-09-12T01:33:47.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.734+0000 7f5acc8af640 1 Processor -- start 2024-09-12T01:33:47.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.734+0000 7f5acc8af640 1 -- start start 2024-09-12T01:33:47.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.735+0000 7f5acc8af640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5ac8156940 0x7f5ac81ef390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:47.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.735+0000 7f5acc8af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815ff00 0x7f5ac80712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:47.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.735+0000 7f5acc8af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8160360 0x7f5ac80717e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:47.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.735+0000 7f5acc8af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac80d1b30 con 0x7f5ac815ff00 2024-09-12T01:33:47.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.735+0000 7f5acc8af640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac80d19b0 con 0x7f5ac8160360 2024-09-12T01:33:47.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.735+0000 7f5acc8af640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac80cfda0 con 0x7f5ac8156940 2024-09-12T01:33:47.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.735+0000 7f5ac5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815ff00 0x7f5ac80712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:47.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.735+0000 7f5ac5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815ff00 0x7f5ac80712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45916/0 (socket says 172.21.15.16:45916) 2024-09-12T01:33:47.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.735+0000 7f5ac5d9b640 1 -- 172.21.15.16:0/2249244968 learned_addr learned my addr 172.21.15.16:0/2249244968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:47.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.735+0000 7f5ac6d9d640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8160360 0x7f5ac80717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:47.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.735+0000 7f5ac659c640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5ac8156940 0x7f5ac81ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:47.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.736+0000 7f5ac5d9b640 1 -- 172.21.15.16:0/2249244968 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5ac8156940 msgr2=0x7f5ac81ef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:47.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.736+0000 7f5ac5d9b640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5ac8156940 0x7f5ac81ef390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:47.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.736+0000 7f5ac5d9b640 1 -- 172.21.15.16:0/2249244968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8160360 msgr2=0x7f5ac80717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:47.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.736+0000 7f5ac5d9b640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8160360 0x7f5ac80717e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:47.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.736+0000 7f5ac5d9b640 1 -- 172.21.15.16:0/2249244968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac81ef8d0 con 0x7f5ac815ff00 2024-09-12T01:33:47.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.736+0000 7f5ac5d9b640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815ff00 0x7f5ac80712a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00ebb0 tx=0x7f5abc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:47.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.736+0000 7f5ab6ffd640 1 -- 172.21.15.16:0/2249244968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc017070 con 0x7f5ac815ff00 2024-09-12T01:33:47.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.736+0000 7f5ab6ffd640 1 -- 172.21.15.16:0/2249244968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5abc00cc30 con 0x7f5ac815ff00 2024-09-12T01:33:47.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.736+0000 7f5acc8af640 1 -- 172.21.15.16:0/2249244968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac81f5540 con 0x7f5ac815ff00 2024-09-12T01:33:47.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.736+0000 7f5ab6ffd640 1 -- 172.21.15.16:0/2249244968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc013470 con 0x7f5ac815ff00 2024-09-12T01:33:47.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.736+0000 7f5acc8af640 1 -- 172.21.15.16:0/2249244968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac81f5b00 con 0x7f5ac815ff00 2024-09-12T01:33:47.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.737+0000 7f5acc8af640 1 -- 172.21.15.16:0/2249244968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac815d320 con 0x7f5ac815ff00 2024-09-12T01:33:47.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.738+0000 7f5ab6ffd640 1 -- 172.21.15.16:0/2249244968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5abc01a070 con 0x7f5ac815ff00 2024-09-12T01:33:47.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.738+0000 7f5ab6ffd640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5aa806be30 0x7f5aa806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:47.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.738+0000 7f5ab6ffd640 1 -- 172.21.15.16:0/2249244968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5abc08b050 con 0x7f5ac815ff00 2024-09-12T01:33:47.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.738+0000 7f5ac659c640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5aa806be30 0x7f5aa806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:47.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.739+0000 7f5ac659c640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5aa806be30 0x7f5aa806e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005880 tx=0x7f5ab8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.743+0000 7f5ab6ffd640 1 -- 172.21.15.16:0/2249244968 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5abc059070 con 0x7f5ac815ff00 2024-09-12T01:33:47.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.919+0000 7f5acc8af640 1 -- 172.21.15.16:0/2249244968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ac815e630 con 0x7f5ac815ff00 2024-09-12T01:33:47.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.920+0000 7f5ab6ffd640 1 -- 172.21.15.16:0/2249244968 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18574 (secure 0 0 0) 0x7f5abc05d370 con 0x7f5ac815ff00 2024-09-12T01:33:47.920 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:47.920 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:33:18.213622+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:33:09.364639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:33:10.478237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:33:14.204624+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:33:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.922+0000 7f5acc8af640 1 -- 172.21.15.16:0/2249244968 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5aa806be30 msgr2=0x7f5aa806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.922+0000 7f5acc8af640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5aa806be30 0x7f5aa806e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005880 tx=0x7f5ab8009040 comp rx=0 tx=0).stop 2024-09-12T01:33:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.922+0000 7f5acc8af640 1 -- 172.21.15.16:0/2249244968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815ff00 msgr2=0x7f5ac80712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.922+0000 7f5acc8af640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815ff00 0x7f5ac80712a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00ebb0 tx=0x7f5abc00c520 comp rx=0 tx=0).stop 2024-09-12T01:33:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.922+0000 7f5acc8af640 1 -- 172.21.15.16:0/2249244968 shutdown_connections 2024-09-12T01:33:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.922+0000 7f5acc8af640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5aa806be30 0x7f5aa806e2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.922+0000 7f5acc8af640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8160360 0x7f5ac80717e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.922+0000 7f5acc8af640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815ff00 0x7f5ac80712a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.922+0000 7f5acc8af640 1 --2- 172.21.15.16:0/2249244968 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5ac8156940 0x7f5ac81ef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.922+0000 7f5acc8af640 1 -- 172.21.15.16:0/2249244968 >> 172.21.15.16:0/2249244968 conn(0x7f5ac806fdc0 msgr2=0x7f5ac80d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.922+0000 7f5acc8af640 1 -- 172.21.15.16:0/2249244968 shutdown_connections 2024-09-12T01:33:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:47.922+0000 7f5acc8af640 1 -- 172.21.15.16:0/2249244968 wait complete. 2024-09-12T01:33:47.933 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T01:32:58.512340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T01:33:09.364639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T01:33:10.478237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-12T01:33:14.204624+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-12T01:33:47.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T01:33:48.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.039+0000 7f64366d0640 1 Processor -- start 2024-09-12T01:33:48.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.039+0000 7f64366d0640 1 -- start start 2024-09-12T01:33:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.039+0000 7f64366d0640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f643015e6a0 0x7f64300d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.039+0000 7f64366d0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64300cf050 con 0x7f64300d3160 2024-09-12T01:33:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.040+0000 7f64366d0640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64300cf2d0 con 0x7f64300d0e30 2024-09-12T01:33:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.040+0000 7f64366d0640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f64300cfc00 con 0x7f643015e6a0 2024-09-12T01:33:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.040+0000 7f642f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f64300d3160 0x7f64300d4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43852/0 (socket says 172.21.15.16:43852) 2024-09-12T01:33:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.040+0000 7f642f7fe640 1 -- 172.21.15.16:0/2802043980 learned_addr learned my addr 172.21.15.16:0/2802043980 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.040+0000 7f642ffff640 1 --2- 172.21.15.16:0/2802043980 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f643015e6a0 0x7f64300d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.040+0000 7f642effd640 1 -- 172.21.15.16:0/2802043980 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 881321890 0 0) 0x7f6424001110 con 0x7f64300d3160 2024-09-12T01:33:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.040+0000 7f642effd640 1 -- 172.21.15.16:0/2802043980 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6418003520 con 0x7f64300d3160 2024-09-12T01:33:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.040+0000 7f642effd640 1 -- 172.21.15.16:0/2802043980 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1951204875 0 0) 0x7f64300cf2d0 con 0x7f64300d0e30 2024-09-12T01:33:48.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.040+0000 7f642effd640 1 -- 172.21.15.16:0/2802043980 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6424001110 con 0x7f64300d0e30 2024-09-12T01:33:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.040+0000 7f642ffff640 1 -- 172.21.15.16:0/2802043980 >> v1:172.21.15.33:6789/0 conn(0x7f64300d0e30 legacy=0x7f64300b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.040+0000 7f642ffff640 1 -- 172.21.15.16:0/2802043980 >> v1:172.21.15.16:6789/0 conn(0x7f64300d3160 legacy=0x7f64300d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.040+0000 7f642ffff640 1 -- 172.21.15.16:0/2802043980 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64300cf5b0 con 0x7f643015e6a0 2024-09-12T01:33:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.041+0000 7f642ffff640 1 --2- 172.21.15.16:0/2802043980 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f643015e6a0 0x7f64300d0090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f64200097c0 tx=0x7f642002a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4617bf5669c7e100 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.041+0000 7f642effd640 1 -- 172.21.15.16:0/2802043980 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420037070 con 0x7f643015e6a0 2024-09-12T01:33:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.041+0000 7f642effd640 1 -- 172.21.15.16:0/2802043980 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6420032440 con 0x7f643015e6a0 2024-09-12T01:33:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.041+0000 7f642effd640 1 -- 172.21.15.16:0/2802043980 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420032810 con 0x7f643015e6a0 2024-09-12T01:33:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.042+0000 7f64366d0640 1 -- 172.21.15.16:0/2802043980 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f643015e6a0 msgr2=0x7f64300d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.042+0000 7f64366d0640 1 --2- 172.21.15.16:0/2802043980 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f643015e6a0 0x7f64300d0090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f64200097c0 tx=0x7f642002a200 comp rx=0 tx=0).stop 2024-09-12T01:33:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.042+0000 7f64366d0640 1 -- 172.21.15.16:0/2802043980 shutdown_connections 2024-09-12T01:33:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.042+0000 7f64366d0640 1 --2- 172.21.15.16:0/2802043980 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f643015e6a0 0x7f64300d0090 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.042+0000 7f64366d0640 1 -- 172.21.15.16:0/2802043980 >> 172.21.15.16:0/2802043980 conn(0x7f643009d730 msgr2=0x7f64300d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.042+0000 7f64366d0640 1 -- 172.21.15.16:0/2802043980 shutdown_connections 2024-09-12T01:33:48.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.042+0000 7f64366d0640 1 -- 172.21.15.16:0/2802043980 wait complete. 2024-09-12T01:33:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.043+0000 7f64366d0640 1 Processor -- start 2024-09-12T01:33:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.043+0000 7f64366d0640 1 -- start start 2024-09-12T01:33:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f64366d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64300d0e30 0x7f6430156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f64366d0640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64300d3160 0x7f643015ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f64366d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643015e6a0 0x7f64300b9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f64366d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64300d1460 con 0x7f64300d0e30 2024-09-12T01:33:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f64366d0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f64300d12e0 con 0x7f643015e6a0 2024-09-12T01:33:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f64366d0640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f64300d15e0 con 0x7f64300d3160 2024-09-12T01:33:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f642ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64300d0e30 0x7f6430156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f642ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64300d0e30 0x7f6430156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45930/0 (socket says 172.21.15.16:45930) 2024-09-12T01:33:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f642ffff640 1 -- 172.21.15.16:0/3882057195 learned_addr learned my addr 172.21.15.16:0/3882057195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f642ffff640 1 -- 172.21.15.16:0/3882057195 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64300d3160 msgr2=0x7f643015ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f642ffff640 1 --2- 172.21.15.16:0/3882057195 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64300d3160 0x7f643015ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f642ffff640 1 -- 172.21.15.16:0/3882057195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643015e6a0 msgr2=0x7f64300b9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f6434c6d640 1 --2- 172.21.15.16:0/3882057195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643015e6a0 0x7f64300b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f642ffff640 1 --2- 172.21.15.16:0/3882057195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643015e6a0 0x7f64300b9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.044+0000 7f642ffff640 1 -- 172.21.15.16:0/3882057195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64301583a0 con 0x7f64300d0e30 2024-09-12T01:33:48.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.045+0000 7f642ffff640 1 --2- 172.21.15.16:0/3882057195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64300d0e30 0x7f6430156e30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6420035040 tx=0x7f64200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:48.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.045+0000 7f642cff9640 1 -- 172.21.15.16:0/3882057195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420037050 con 0x7f64300d0e30 2024-09-12T01:33:48.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.045+0000 7f642cff9640 1 -- 172.21.15.16:0/3882057195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6420043070 con 0x7f64300d0e30 2024-09-12T01:33:48.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.045+0000 7f64366d0640 1 -- 172.21.15.16:0/3882057195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64300c81a0 con 0x7f64300d0e30 2024-09-12T01:33:48.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.045+0000 7f642cff9640 1 -- 172.21.15.16:0/3882057195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642003cb30 con 0x7f64300d0e30 2024-09-12T01:33:48.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.045+0000 7f64366d0640 1 -- 172.21.15.16:0/3882057195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64300c8760 con 0x7f64300d0e30 2024-09-12T01:33:48.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.046+0000 7f64366d0640 1 -- 172.21.15.16:0/3882057195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64300d0090 con 0x7f64300d0e30 2024-09-12T01:33:48.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.046+0000 7f642cff9640 1 -- 172.21.15.16:0/3882057195 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f642003ccd0 con 0x7f64300d0e30 2024-09-12T01:33:48.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.047+0000 7f642cff9640 1 --2- 172.21.15.16:0/3882057195 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f641c06be30 0x7f641c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.047+0000 7f642cff9640 1 -- 172.21.15.16:0/3882057195 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f64200abf20 con 0x7f64300d0e30 2024-09-12T01:33:48.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.047+0000 7f642f7fe640 1 --2- 172.21.15.16:0/3882057195 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f641c06be30 0x7f641c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.047+0000 7f642f7fe640 1 --2- 172.21.15.16:0/3882057195 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f641c06be30 0x7f641c06e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6424005880 tx=0x7f6424007950 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:48.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.051+0000 7f642cff9640 1 -- 172.21.15.16:0/3882057195 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f64301583a0 con 0x7f64300d0e30 2024-09-12T01:33:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.229+0000 7f64366d0640 1 -- 172.21.15.16:0/3882057195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f643006dde0 con 0x7f64300d0e30 2024-09-12T01:33:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.229+0000 7f642cff9640 1 -- 172.21.15.16:0/3882057195 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f6420046360 con 0x7f64300d0e30 2024-09-12T01:33:48.229 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-12T01:33:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.231+0000 7f64366d0640 1 -- 172.21.15.16:0/3882057195 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f641c06be30 msgr2=0x7f641c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.231+0000 7f64366d0640 1 --2- 172.21.15.16:0/3882057195 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f641c06be30 0x7f641c06e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6424005880 tx=0x7f6424007950 comp rx=0 tx=0).stop 2024-09-12T01:33:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.231+0000 7f64366d0640 1 -- 172.21.15.16:0/3882057195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64300d0e30 msgr2=0x7f6430156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.231+0000 7f64366d0640 1 --2- 172.21.15.16:0/3882057195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64300d0e30 0x7f6430156e30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6420035040 tx=0x7f64200046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.231+0000 7f64366d0640 1 -- 172.21.15.16:0/3882057195 shutdown_connections 2024-09-12T01:33:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.231+0000 7f64366d0640 1 --2- 172.21.15.16:0/3882057195 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f641c06be30 0x7f641c06e2f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.231+0000 7f64366d0640 1 --2- 172.21.15.16:0/3882057195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643015e6a0 0x7f64300b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.231+0000 7f64366d0640 1 --2- 172.21.15.16:0/3882057195 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64300d3160 0x7f643015ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.231+0000 7f64366d0640 1 --2- 172.21.15.16:0/3882057195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64300d0e30 0x7f6430156e30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.231+0000 7f64366d0640 1 -- 172.21.15.16:0/3882057195 >> 172.21.15.16:0/3882057195 conn(0x7f643009d730 msgr2=0x7f64301600a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.231+0000 7f64366d0640 1 -- 172.21.15.16:0/3882057195 shutdown_connections 2024-09-12T01:33:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.231+0000 7f64366d0640 1 -- 172.21.15.16:0/3882057195 wait complete. 2024-09-12T01:33:48.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T01:33:48.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.348+0000 7f04e1942640 1 Processor -- start 2024-09-12T01:33:48.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.348+0000 7f04e1942640 1 -- start start 2024-09-12T01:33:48.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04e1942640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f04dc15e6a0 0x7f04dc0d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04e1942640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04dc0d0e30 0x7f04dc0d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04e1942640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04dc0cf050 con 0x7f04dc0d3160 2024-09-12T01:33:48.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04e1942640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f04dc0cfc00 con 0x7f04dc0d0e30 2024-09-12T01:33:48.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04e1942640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f04dc0cf2d0 con 0x7f04dc15e6a0 2024-09-12T01:33:48.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04db7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04dc0d0e30 0x7f04dc0d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04da7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f04dc0d3160 0x7f04dc0d4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43864/0 (socket says 172.21.15.16:43864) 2024-09-12T01:33:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04da7fc640 1 -- 172.21.15.16:0/1029478194 learned_addr learned my addr 172.21.15.16:0/1029478194 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04db7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04dc0d0e30 0x7f04dc0d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45942/0 (socket says 172.21.15.16:45942) 2024-09-12T01:33:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04daffd640 1 --2- 172.21.15.16:0/1029478194 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f04dc15e6a0 0x7f04dc0d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04db7fe640 1 -- 172.21.15.16:0/1029478194 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f04dc15e6a0 msgr2=0x7f04dc0d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04db7fe640 1 --2- 172.21.15.16:0/1029478194 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f04dc15e6a0 0x7f04dc0d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04db7fe640 1 -- 172.21.15.16:0/1029478194 >> v1:172.21.15.16:6789/0 conn(0x7f04dc0d3160 legacy=0x7f04dc0d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04db7fe640 1 -- 172.21.15.16:0/1029478194 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc0cf5b0 con 0x7f04dc0d0e30 2024-09-12T01:33:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04d9ffb640 1 -- 172.21.15.16:0/1029478194 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2939265433 0 0) 0x7f04d0001110 con 0x7f04dc0d3160 2024-09-12T01:33:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.349+0000 7f04daffd640 1 --2- 172.21.15.16:0/1029478194 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f04dc15e6a0 0x7f04dc0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.350+0000 7f04db7fe640 1 --2- 172.21.15.16:0/1029478194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04dc0d0e30 0x7f04dc0d12e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009d90 tx=0x7f04d402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0407f1153c0198 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.350+0000 7f04d9ffb640 1 -- 172.21.15.16:0/1029478194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4007740 con 0x7f04dc0d0e30 2024-09-12T01:33:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.350+0000 7f04d9ffb640 1 -- 172.21.15.16:0/1029478194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d40078e0 con 0x7f04dc0d0e30 2024-09-12T01:33:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.350+0000 7f04d9ffb640 1 -- 172.21.15.16:0/1029478194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4007cb0 con 0x7f04dc0d0e30 2024-09-12T01:33:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.351+0000 7f04e1942640 1 -- 172.21.15.16:0/1029478194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04dc0d0e30 msgr2=0x7f04dc0d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.351+0000 7f04e1942640 1 --2- 172.21.15.16:0/1029478194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04dc0d0e30 0x7f04dc0d12e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009d90 tx=0x7f04d402a660 comp rx=0 tx=0).stop 2024-09-12T01:33:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.351+0000 7f04e1942640 1 -- 172.21.15.16:0/1029478194 shutdown_connections 2024-09-12T01:33:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.351+0000 7f04e1942640 1 --2- 172.21.15.16:0/1029478194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04dc0d0e30 0x7f04dc0d12e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.351+0000 7f04e1942640 1 --2- 172.21.15.16:0/1029478194 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f04dc15e6a0 0x7f04dc0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.351+0000 7f04e1942640 1 -- 172.21.15.16:0/1029478194 >> 172.21.15.16:0/1029478194 conn(0x7f04dc09d730 msgr2=0x7f04dc0d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.351+0000 7f04e1942640 1 -- 172.21.15.16:0/1029478194 shutdown_connections 2024-09-12T01:33:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.351+0000 7f04e1942640 1 -- 172.21.15.16:0/1029478194 wait complete. 2024-09-12T01:33:48.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.352+0000 7f04e1942640 1 Processor -- start 2024-09-12T01:33:48.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.352+0000 7f04e1942640 1 -- start start 2024-09-12T01:33:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.353+0000 7f04e1942640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc0d0e30 0x7f04dc156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.353+0000 7f04e1942640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f04dc0d3160 0x7f04dc0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.353+0000 7f04e1942640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04dc15e6a0 0x7f04dc1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.353+0000 7f04e1942640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc0b73f0 con 0x7f04dc15e6a0 2024-09-12T01:33:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.353+0000 7f04e1942640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc0b7270 con 0x7f04dc0d0e30 2024-09-12T01:33:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.353+0000 7f04e1942640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f04dc0b7570 con 0x7f04dc0d3160 2024-09-12T01:33:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.353+0000 7f04db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04dc15e6a0 0x7f04dc1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.353+0000 7f04db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04dc15e6a0 0x7f04dc1589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45956/0 (socket says 172.21.15.16:45956) 2024-09-12T01:33:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.353+0000 7f04db7fe640 1 -- 172.21.15.16:0/3010644606 learned_addr learned my addr 172.21.15.16:0/3010644606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.353+0000 7f04daffd640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc0d0e30 0x7f04dc156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.353+0000 7f04da7fc640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f04dc0d3160 0x7f04dc0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.354+0000 7f04db7fe640 1 -- 172.21.15.16:0/3010644606 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f04dc0d3160 msgr2=0x7f04dc0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.354+0000 7f04db7fe640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f04dc0d3160 0x7f04dc0cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.354+0000 7f04db7fe640 1 -- 172.21.15.16:0/3010644606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc0d0e30 msgr2=0x7f04dc156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.354+0000 7f04db7fe640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc0d0e30 0x7f04dc156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.354+0000 7f04db7fe640 1 -- 172.21.15.16:0/3010644606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc0cdab0 con 0x7f04dc15e6a0 2024-09-12T01:33:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.354+0000 7f04daffd640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc0d0e30 0x7f04dc156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.354+0000 7f04da7fc640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f04dc0d3160 0x7f04dc0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.354+0000 7f04db7fe640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04dc15e6a0 0x7f04dc1589b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009d90 tx=0x7f04d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.354+0000 7f04c3fff640 1 -- 172.21.15.16:0/3010644606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4040070 con 0x7f04dc15e6a0 2024-09-12T01:33:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.354+0000 7f04c3fff640 1 -- 172.21.15.16:0/3010644606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d4031c40 con 0x7f04dc15e6a0 2024-09-12T01:33:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.354+0000 7f04e1942640 1 -- 172.21.15.16:0/3010644606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc0c8160 con 0x7f04dc15e6a0 2024-09-12T01:33:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.354+0000 7f04c3fff640 1 -- 172.21.15.16:0/3010644606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4003990 con 0x7f04dc15e6a0 2024-09-12T01:33:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.354+0000 7f04e1942640 1 -- 172.21.15.16:0/3010644606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc0c8720 con 0x7f04dc15e6a0 2024-09-12T01:33:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.355+0000 7f04e1942640 1 -- 172.21.15.16:0/3010644606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04dc0c7ed0 con 0x7f04dc15e6a0 2024-09-12T01:33:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.356+0000 7f04c3fff640 1 -- 172.21.15.16:0/3010644606 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04d4039040 con 0x7f04dc15e6a0 2024-09-12T01:33:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.356+0000 7f04c3fff640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f04c806be30 0x7f04c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.356+0000 7f04daffd640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f04c806be30 0x7f04c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.356+0000 7f04c3fff640 1 -- 172.21.15.16:0/3010644606 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f04d40ac580 con 0x7f04dc15e6a0 2024-09-12T01:33:48.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.357+0000 7f04daffd640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f04c806be30 0x7f04c806e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f04cc008870 tx=0x7f04cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.361+0000 7f04c3fff640 1 -- 172.21.15.16:0/3010644606 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f04d407ab60 con 0x7f04dc15e6a0 2024-09-12T01:33:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.537+0000 7f04e1942640 1 -- 172.21.15.16:0/3010644606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f04dc0d0c10 con 0x7f04dc15e6a0 2024-09-12T01:33:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.537+0000 7f04c3fff640 1 -- 172.21.15.16:0/3010644606 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f04d407a460 con 0x7f04dc15e6a0 2024-09-12T01:33:48.537 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-12T01:33:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.539+0000 7f04e1942640 1 -- 172.21.15.16:0/3010644606 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f04c806be30 msgr2=0x7f04c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.539+0000 7f04e1942640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f04c806be30 0x7f04c806e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f04cc008870 tx=0x7f04cc008040 comp rx=0 tx=0).stop 2024-09-12T01:33:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.539+0000 7f04e1942640 1 -- 172.21.15.16:0/3010644606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04dc15e6a0 msgr2=0x7f04dc1589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.539+0000 7f04e1942640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04dc15e6a0 0x7f04dc1589b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009d90 tx=0x7f04d4004a40 comp rx=0 tx=0).stop 2024-09-12T01:33:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.539+0000 7f04e1942640 1 -- 172.21.15.16:0/3010644606 shutdown_connections 2024-09-12T01:33:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.539+0000 7f04e1942640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f04c806be30 0x7f04c806e2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.539+0000 7f04e1942640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04dc15e6a0 0x7f04dc1589b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.539+0000 7f04e1942640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f04dc0d3160 0x7f04dc0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.539+0000 7f04e1942640 1 --2- 172.21.15.16:0/3010644606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc0d0e30 0x7f04dc156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.539+0000 7f04e1942640 1 -- 172.21.15.16:0/3010644606 >> 172.21.15.16:0/3010644606 conn(0x7f04dc09d730 msgr2=0x7f04dc0d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.539+0000 7f04e1942640 1 -- 172.21.15.16:0/3010644606 shutdown_connections 2024-09-12T01:33:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.539+0000 7f04e1942640 1 -- 172.21.15.16:0/3010644606 wait complete. 2024-09-12T01:33:48.550 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T01:33:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.656+0000 7f50ee447640 1 Processor -- start 2024-09-12T01:33:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.656+0000 7f50ee447640 1 -- start start 2024-09-12T01:33:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.657+0000 7f50ee447640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f50e80d2830 0x7f50e81602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.657+0000 7f50ee447640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f50e80b7bc0 0x7f50e80d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.657+0000 7f50ee447640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50e8160a30 con 0x7f50e815fe70 2024-09-12T01:33:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.657+0000 7f50ee447640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f50e8160800 con 0x7f50e80d2830 2024-09-12T01:33:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.657+0000 7f50ee447640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f50e80d0350 con 0x7f50e80b7bc0 2024-09-12T01:33:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.657+0000 7f50ec9e4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f50e80b7bc0 0x7f50e80d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.657+0000 7f50e77fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f50e80d2830 0x7f50e81602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.657+0000 7f50ec9e4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f50e80b7bc0 0x7f50e80d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:49574/0 (socket says 172.21.15.16:49574) 2024-09-12T01:33:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.657+0000 7f50e7fff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f50e815fe70 0x7f50e80d2120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:49794/0 (socket says 172.21.15.16:49794) 2024-09-12T01:33:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.657+0000 7f50ec9e4640 1 -- 172.21.15.16:0/4186106318 learned_addr learned my addr 172.21.15.16:0/4186106318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.658+0000 7f50e6ffd640 1 -- 172.21.15.16:0/4186106318 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 447032410 0 0) 0x7f50e8160a30 con 0x7f50e815fe70 2024-09-12T01:33:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.658+0000 7f50e6ffd640 1 -- 172.21.15.16:0/4186106318 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50d0003520 con 0x7f50e815fe70 2024-09-12T01:33:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.658+0000 7f50ec9e4640 1 -- 172.21.15.16:0/4186106318 >> v2:172.21.15.33:3300/0 conn(0x7f50e80d2830 msgr2=0x7f50e81602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.658+0000 7f50ec9e4640 1 --2- 172.21.15.16:0/4186106318 >> v2:172.21.15.33:3300/0 conn(0x7f50e80d2830 0x7f50e81602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.658+0000 7f50ec9e4640 1 -- 172.21.15.16:0/4186106318 >> v1:172.21.15.33:6789/0 conn(0x7f50e815fe70 legacy=0x7f50e80d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.658+0000 7f50ec9e4640 1 -- 172.21.15.16:0/4186106318 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e8160d10 con 0x7f50e80b7bc0 2024-09-12T01:33:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.658+0000 7f50e77fe640 1 --2- 172.21.15.16:0/4186106318 >> v2:172.21.15.33:3300/0 conn(0x7f50e80d2830 0x7f50e81602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.658+0000 7f50ec9e4640 1 --2- 172.21.15.16:0/4186106318 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f50e80b7bc0 0x7f50e80d07e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009e50 tx=0x7f50e002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fab0bf40e90a8d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.659+0000 7f50e6ffd640 1 -- 172.21.15.16:0/4186106318 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0036070 con 0x7f50e80b7bc0 2024-09-12T01:33:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.659+0000 7f50e6ffd640 1 -- 172.21.15.16:0/4186106318 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e0031600 con 0x7f50e80b7bc0 2024-09-12T01:33:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.659+0000 7f50e6ffd640 1 -- 172.21.15.16:0/4186106318 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e00319b0 con 0x7f50e80b7bc0 2024-09-12T01:33:48.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.659+0000 7f50ee447640 1 -- 172.21.15.16:0/4186106318 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f50e80b7bc0 msgr2=0x7f50e80d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.659+0000 7f50ee447640 1 --2- 172.21.15.16:0/4186106318 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f50e80b7bc0 0x7f50e80d07e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009e50 tx=0x7f50e002a680 comp rx=0 tx=0).stop 2024-09-12T01:33:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.659+0000 7f50ee447640 1 -- 172.21.15.16:0/4186106318 shutdown_connections 2024-09-12T01:33:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.659+0000 7f50ee447640 1 --2- 172.21.15.16:0/4186106318 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f50e80b7bc0 0x7f50e80d07e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.659+0000 7f50ee447640 1 --2- 172.21.15.16:0/4186106318 >> v2:172.21.15.33:3300/0 conn(0x7f50e80d2830 0x7f50e81602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.659+0000 7f50ee447640 1 -- 172.21.15.16:0/4186106318 >> 172.21.15.16:0/4186106318 conn(0x7f50e806fed0 msgr2=0x7f50e80702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.659+0000 7f50ee447640 1 -- 172.21.15.16:0/4186106318 shutdown_connections 2024-09-12T01:33:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.659+0000 7f50ee447640 1 -- 172.21.15.16:0/4186106318 wait complete. 2024-09-12T01:33:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.661+0000 7f50ee447640 1 Processor -- start 2024-09-12T01:33:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.661+0000 7f50ee447640 1 -- start start 2024-09-12T01:33:48.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.661+0000 7f50ee447640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e80b7bc0 0x7f50e815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.661+0000 7f50ee447640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f50e80d2830 0x7f50e8158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.661+0000 7f50ee447640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e815fe70 0x7f50e80a9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.661+0000 7f50ee447640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50e80d0ea0 con 0x7f50e80b7bc0 2024-09-12T01:33:48.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.661+0000 7f50ee447640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50e80d0d20 con 0x7f50e815fe70 2024-09-12T01:33:48.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.661+0000 7f50ee447640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f50e80d1020 con 0x7f50e80d2830 2024-09-12T01:33:48.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e80b7bc0 0x7f50e815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e80b7bc0 0x7f50e815c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45962/0 (socket says 172.21.15.16:45962) 2024-09-12T01:33:48.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50e7fff640 1 -- 172.21.15.16:0/1763029086 learned_addr learned my addr 172.21.15.16:0/1763029086 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:48.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50e77fe640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f50e80d2830 0x7f50e8158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50ec9e4640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e815fe70 0x7f50e80a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50e7fff640 1 -- 172.21.15.16:0/1763029086 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f50e80d2830 msgr2=0x7f50e8158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50e7fff640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f50e80d2830 0x7f50e8158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50e7fff640 1 -- 172.21.15.16:0/1763029086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e815fe70 msgr2=0x7f50e80a9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50e7fff640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e815fe70 0x7f50e80a9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50e7fff640 1 -- 172.21.15.16:0/1763029086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e815c8e0 con 0x7f50e80b7bc0 2024-09-12T01:33:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50e7fff640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e80b7bc0 0x7f50e815c1b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c390 tx=0x7f50d800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50e4ff9640 1 -- 172.21.15.16:0/1763029086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d80074e0 con 0x7f50e80b7bc0 2024-09-12T01:33:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50e4ff9640 1 -- 172.21.15.16:0/1763029086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d80047e0 con 0x7f50e80b7bc0 2024-09-12T01:33:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50ee447640 1 -- 172.21.15.16:0/1763029086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e80c77a0 con 0x7f50e80b7bc0 2024-09-12T01:33:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50e4ff9640 1 -- 172.21.15.16:0/1763029086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8004bd0 con 0x7f50e80b7bc0 2024-09-12T01:33:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.662+0000 7f50ee447640 1 -- 172.21.15.16:0/1763029086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e80c7d60 con 0x7f50e80b7bc0 2024-09-12T01:33:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.663+0000 7f50ee447640 1 -- 172.21.15.16:0/1763029086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50e80d07e0 con 0x7f50e80b7bc0 2024-09-12T01:33:48.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.664+0000 7f50e4ff9640 1 -- 172.21.15.16:0/1763029086 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50d801a020 con 0x7f50e80b7bc0 2024-09-12T01:33:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.664+0000 7f50e4ff9640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f50d406be30 0x7f50d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.665+0000 7f50e4ff9640 1 -- 172.21.15.16:0/1763029086 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f50d808a7f0 con 0x7f50e80b7bc0 2024-09-12T01:33:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.665+0000 7f50e77fe640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f50d406be30 0x7f50d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.665+0000 7f50e77fe640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f50d406be30 0x7f50d406e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f50e815e100 tx=0x7f50dc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.669+0000 7f50e4ff9640 1 -- 172.21.15.16:0/1763029086 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f50d80588d0 con 0x7f50e80b7bc0 2024-09-12T01:33:48.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.844+0000 7f50ee447640 1 -- 172.21.15.16:0/1763029086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f50e806ee10 con 0x7f50e80b7bc0 2024-09-12T01:33:48.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.844+0000 7f50e4ff9640 1 -- 172.21.15.16:0/1763029086 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f50d805cbd0 con 0x7f50e80b7bc0 2024-09-12T01:33:48.844 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-12T01:33:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.846+0000 7f50ee447640 1 -- 172.21.15.16:0/1763029086 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f50d406be30 msgr2=0x7f50d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.846+0000 7f50ee447640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f50d406be30 0x7f50d406e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f50e815e100 tx=0x7f50dc009190 comp rx=0 tx=0).stop 2024-09-12T01:33:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.846+0000 7f50ee447640 1 -- 172.21.15.16:0/1763029086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e80b7bc0 msgr2=0x7f50e815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.846+0000 7f50ee447640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e80b7bc0 0x7f50e815c1b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c390 tx=0x7f50d800c850 comp rx=0 tx=0).stop 2024-09-12T01:33:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.847+0000 7f50ee447640 1 -- 172.21.15.16:0/1763029086 shutdown_connections 2024-09-12T01:33:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.847+0000 7f50ee447640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f50d406be30 0x7f50d406e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.847+0000 7f50ee447640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e815fe70 0x7f50e80a9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.847+0000 7f50ee447640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f50e80d2830 0x7f50e8158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.847+0000 7f50ee447640 1 --2- 172.21.15.16:0/1763029086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e80b7bc0 0x7f50e815c1b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.847+0000 7f50ee447640 1 -- 172.21.15.16:0/1763029086 >> 172.21.15.16:0/1763029086 conn(0x7f50e806fed0 msgr2=0x7f50e80d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.847+0000 7f50ee447640 1 -- 172.21.15.16:0/1763029086 shutdown_connections 2024-09-12T01:33:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.847+0000 7f50ee447640 1 -- 172.21.15.16:0/1763029086 wait complete. 2024-09-12T01:33:48.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-12T01:33:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.963+0000 7f0c1f9c6640 1 Processor -- start 2024-09-12T01:33:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.963+0000 7f0c1f9c6640 1 -- start start 2024-09-12T01:33:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.964+0000 7f0c1f9c6640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0c180d1540 0x7f0c180d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.964+0000 7f0c1f9c6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c180cf050 con 0x7f0c1815e6a0 2024-09-12T01:33:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.964+0000 7f0c1f9c6640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c180cf2d0 con 0x7f0c180d0090 2024-09-12T01:33:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.964+0000 7f0c1f9c6640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0c180cfc00 con 0x7f0c180d1540 2024-09-12T01:33:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.964+0000 7f0c1d762640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0c1815e6a0 0x7f0c180d4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43870/0 (socket says 172.21.15.16:43870) 2024-09-12T01:33:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.964+0000 7f0c1d762640 1 -- 172.21.15.16:0/1526572585 learned_addr learned my addr 172.21.15.16:0/1526572585 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.964+0000 7f0c1df63640 1 --2- 172.21.15.16:0/1526572585 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0c180d1540 0x7f0c180d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.964+0000 7f0c0ffff640 1 -- 172.21.15.16:0/1526572585 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3484455776 0 0) 0x7f0c14001290 con 0x7f0c1815e6a0 2024-09-12T01:33:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.964+0000 7f0c0ffff640 1 -- 172.21.15.16:0/1526572585 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c04003520 con 0x7f0c1815e6a0 2024-09-12T01:33:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.964+0000 7f0c1df63640 1 -- 172.21.15.16:0/1526572585 >> v1:172.21.15.33:6789/0 conn(0x7f0c180d0090 legacy=0x7f0c180d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.964+0000 7f0c0ffff640 1 -- 172.21.15.16:0/1526572585 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4206010352 0 0) 0x7f0c180cf2d0 con 0x7f0c180d0090 2024-09-12T01:33:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.964+0000 7f0c1df63640 1 -- 172.21.15.16:0/1526572585 >> v1:172.21.15.16:6789/0 conn(0x7f0c1815e6a0 legacy=0x7f0c180d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.964+0000 7f0c1df63640 1 -- 172.21.15.16:0/1526572585 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c180cf5b0 con 0x7f0c180d1540 2024-09-12T01:33:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.965+0000 7f0c1df63640 1 --2- 172.21.15.16:0/1526572585 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0c180d1540 0x7f0c180d3160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009dc0 tx=0x7f0c1002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd30210d50d645a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.965+0000 7f0c0ffff640 1 -- 172.21.15.16:0/1526572585 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10007740 con 0x7f0c180d1540 2024-09-12T01:33:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.965+0000 7f0c0ffff640 1 -- 172.21.15.16:0/1526572585 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c100078e0 con 0x7f0c180d1540 2024-09-12T01:33:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.965+0000 7f0c0ffff640 1 -- 172.21.15.16:0/1526572585 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10007cb0 con 0x7f0c180d1540 2024-09-12T01:33:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.966+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/1526572585 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0c180d1540 msgr2=0x7f0c180d3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.966+0000 7f0c1f9c6640 1 --2- 172.21.15.16:0/1526572585 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0c180d1540 0x7f0c180d3160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009dc0 tx=0x7f0c1002a6d0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.966+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/1526572585 shutdown_connections 2024-09-12T01:33:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.966+0000 7f0c1f9c6640 1 --2- 172.21.15.16:0/1526572585 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0c180d1540 0x7f0c180d3160 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.966+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/1526572585 >> 172.21.15.16:0/1526572585 conn(0x7f0c1809d730 msgr2=0x7f0c180d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.966+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/1526572585 shutdown_connections 2024-09-12T01:33:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.966+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/1526572585 wait complete. 2024-09-12T01:33:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.967+0000 7f0c1f9c6640 1 Processor -- start 2024-09-12T01:33:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.967+0000 7f0c1f9c6640 1 -- start start 2024-09-12T01:33:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1f9c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c180d0090 0x7f0c180cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1f9c6640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0c180d1540 0x7f0c180cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1f9c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1815e6a0 0x7f0c181f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1f9c6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c180b73f0 con 0x7f0c180d0090 2024-09-12T01:33:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1f9c6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c180b7270 con 0x7f0c1815e6a0 2024-09-12T01:33:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1f9c6640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0c180b7570 con 0x7f0c180d1540 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1d762640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c180d0090 0x7f0c180cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1d762640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c180d0090 0x7f0c180cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45974/0 (socket says 172.21.15.16:45974) 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1d762640 1 -- 172.21.15.16:0/3551492341 learned_addr learned my addr 172.21.15.16:0/3551492341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1cf61640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0c180d1540 0x7f0c180cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1df63640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1815e6a0 0x7f0c181f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1d762640 1 -- 172.21.15.16:0/3551492341 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0c180d1540 msgr2=0x7f0c180cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1d762640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0c180d1540 0x7f0c180cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1d762640 1 -- 172.21.15.16:0/3551492341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1815e6a0 msgr2=0x7f0c181f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1d762640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1815e6a0 0x7f0c181f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1d762640 1 -- 172.21.15.16:0/3551492341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c180c8590 con 0x7f0c180d0090 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.968+0000 7f0c1d762640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c180d0090 0x7f0c180cb5d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400c370 tx=0x7f0c1400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.969+0000 7f0c0dffb640 1 -- 172.21.15.16:0/3551492341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14017070 con 0x7f0c180d0090 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.969+0000 7f0c0dffb640 1 -- 172.21.15.16:0/3551492341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c140074e0 con 0x7f0c180d0090 2024-09-12T01:33:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.969+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/3551492341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c180ce400 con 0x7f0c180d0090 2024-09-12T01:33:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.969+0000 7f0c0dffb640 1 -- 172.21.15.16:0/3551492341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c140059c0 con 0x7f0c180d0090 2024-09-12T01:33:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.969+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/3551492341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c180c8850 con 0x7f0c180d0090 2024-09-12T01:33:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.970+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/3551492341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c180d3160 con 0x7f0c180d0090 2024-09-12T01:33:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.970+0000 7f0c0dffb640 1 -- 172.21.15.16:0/3551492341 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c1401a070 con 0x7f0c180d0090 2024-09-12T01:33:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.971+0000 7f0c0dffb640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0bfc06be30 0x7f0bfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.971+0000 7f0c0dffb640 1 -- 172.21.15.16:0/3551492341 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0c1408a4b0 con 0x7f0c180d0090 2024-09-12T01:33:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.971+0000 7f0c1cf61640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0bfc06be30 0x7f0bfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.971+0000 7f0c1cf61640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0bfc06be30 0x7f0bfc06e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0c18158690 tx=0x7f0c08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:48.975+0000 7f0c0dffb640 1 -- 172.21.15.16:0/3551492341 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0c180cf050 con 0x7f0c180d0090 2024-09-12T01:33:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.150+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/3551492341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0c180d0c10 con 0x7f0c180d0090 2024-09-12T01:33:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.150+0000 7f0c0dffb640 1 -- 172.21.15.16:0/3551492341 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f0c140584c0 con 0x7f0c180d0090 2024-09-12T01:33:49.150 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-12T01:33:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.152+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/3551492341 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0bfc06be30 msgr2=0x7f0bfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.152+0000 7f0c1f9c6640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0bfc06be30 0x7f0bfc06e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0c18158690 tx=0x7f0c08009040 comp rx=0 tx=0).stop 2024-09-12T01:33:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.152+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/3551492341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c180d0090 msgr2=0x7f0c180cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.152+0000 7f0c1f9c6640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c180d0090 0x7f0c180cb5d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400c370 tx=0x7f0c1400c830 comp rx=0 tx=0).stop 2024-09-12T01:33:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.152+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/3551492341 shutdown_connections 2024-09-12T01:33:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.152+0000 7f0c1f9c6640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0bfc06be30 0x7f0bfc06e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.152+0000 7f0c1f9c6640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1815e6a0 0x7f0c181f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.152+0000 7f0c1f9c6640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0c180d1540 0x7f0c180cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.152+0000 7f0c1f9c6640 1 --2- 172.21.15.16:0/3551492341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c180d0090 0x7f0c180cb5d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.152+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/3551492341 >> 172.21.15.16:0/3551492341 conn(0x7f0c1809d730 msgr2=0x7f0c180b85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.152+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/3551492341 shutdown_connections 2024-09-12T01:33:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.152+0000 7f0c1f9c6640 1 -- 172.21.15.16:0/3551492341 wait complete. 2024-09-12T01:33:49.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create' 2024-09-12T01:33:49.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.265+0000 7f197f96d640 1 Processor -- start 2024-09-12T01:33:49.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.265+0000 7f197f96d640 1 -- start start 2024-09-12T01:33:49.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.266+0000 7f197f96d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19780cc540 0x7f19780af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:49.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.266+0000 7f197f96d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19780cb1a0 con 0x7f197815c2b0 2024-09-12T01:33:49.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.266+0000 7f197f96d640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1978071c70 con 0x7f19780cb930 2024-09-12T01:33:49.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.266+0000 7f197f96d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f19780cb480 con 0x7f19780cc540 2024-09-12T01:33:49.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.266+0000 7f197cf08640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19780cc540 0x7f19780af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:49.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.266+0000 7f197d709640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f197815c2b0 0x7f19780738c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43876/0 (socket says 172.21.15.16:43876) 2024-09-12T01:33:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.266+0000 7f197d709640 1 -- 172.21.15.16:0/2989958034 learned_addr learned my addr 172.21.15.16:0/2989958034 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.266+0000 7f197cf08640 1 -- 172.21.15.16:0/2989958034 >> v1:172.21.15.33:6789/0 conn(0x7f19780cb930 legacy=0x7f19780acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.266+0000 7f197cf08640 1 -- 172.21.15.16:0/2989958034 >> v1:172.21.15.16:6789/0 conn(0x7f197815c2b0 legacy=0x7f19780738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.267+0000 7f197cf08640 1 -- 172.21.15.16:0/2989958034 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19780aec80 con 0x7f19780cc540 2024-09-12T01:33:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.266+0000 7f196ffff640 1 -- 172.21.15.16:0/2989958034 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3225444766 0 0) 0x7f19780cb1a0 con 0x7f197815c2b0 2024-09-12T01:33:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.267+0000 7f197cf08640 1 --2- 172.21.15.16:0/2989958034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19780cc540 0x7f19780af060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f19680097a0 tx=0x7f196802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f3dd7130cf28b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.267+0000 7f196ffff640 1 -- 172.21.15.16:0/2989958034 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968037070 con 0x7f19780cc540 2024-09-12T01:33:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.267+0000 7f196ffff640 1 -- 172.21.15.16:0/2989958034 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1968032440 con 0x7f19780cc540 2024-09-12T01:33:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.267+0000 7f196ffff640 1 -- 172.21.15.16:0/2989958034 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968032810 con 0x7f19780cc540 2024-09-12T01:33:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.268+0000 7f197f96d640 1 -- 172.21.15.16:0/2989958034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19780cc540 msgr2=0x7f19780af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.268+0000 7f197f96d640 1 --2- 172.21.15.16:0/2989958034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19780cc540 0x7f19780af060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f19680097a0 tx=0x7f196802a230 comp rx=0 tx=0).stop 2024-09-12T01:33:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.268+0000 7f197f96d640 1 -- 172.21.15.16:0/2989958034 shutdown_connections 2024-09-12T01:33:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.268+0000 7f197f96d640 1 --2- 172.21.15.16:0/2989958034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19780cc540 0x7f19780af060 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.268+0000 7f197f96d640 1 -- 172.21.15.16:0/2989958034 >> 172.21.15.16:0/2989958034 conn(0x7f19780cffa0 msgr2=0x7f19780d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.268+0000 7f197f96d640 1 -- 172.21.15.16:0/2989958034 shutdown_connections 2024-09-12T01:33:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.268+0000 7f197f96d640 1 -- 172.21.15.16:0/2989958034 wait complete. 2024-09-12T01:33:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.269+0000 7f197f96d640 1 Processor -- start 2024-09-12T01:33:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.269+0000 7f197f96d640 1 -- start start 2024-09-12T01:33:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197f96d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19780cb930 0x7f19780b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197f96d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19780cc540 0x7f19780b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197f96d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f197815c2b0 0x7f197809c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197f96d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19780ad8f0 con 0x7f19780cb930 2024-09-12T01:33:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197f96d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19780ad770 con 0x7f19780cc540 2024-09-12T01:33:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197f96d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f19780cbdc0 con 0x7f197815c2b0 2024-09-12T01:33:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197d709640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19780cb930 0x7f19780b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197d709640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19780cb930 0x7f19780b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45998/0 (socket says 172.21.15.16:45998) 2024-09-12T01:33:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197d709640 1 -- 172.21.15.16:0/780357230 learned_addr learned my addr 172.21.15.16:0/780357230 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197cf08640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19780cc540 0x7f19780b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197df0a640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f197815c2b0 0x7f197809c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197d709640 1 -- 172.21.15.16:0/780357230 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f197815c2b0 msgr2=0x7f197809c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197d709640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f197815c2b0 0x7f197809c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197d709640 1 -- 172.21.15.16:0/780357230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19780cc540 msgr2=0x7f19780b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197d709640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19780cc540 0x7f19780b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197d709640 1 -- 172.21.15.16:0/780357230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19780b4850 con 0x7f19780cb930 2024-09-12T01:33:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197cf08640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19780cc540 0x7f19780b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.270+0000 7f197df0a640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f197815c2b0 0x7f197809c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.271+0000 7f197d709640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19780cb930 0x7f19780b0bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f197400c3f0 tx=0x7f197400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.271+0000 7f196dffb640 1 -- 172.21.15.16:0/780357230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740074e0 con 0x7f19780cb930 2024-09-12T01:33:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.271+0000 7f196dffb640 1 -- 172.21.15.16:0/780357230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19740047e0 con 0x7f19780cb930 2024-09-12T01:33:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.271+0000 7f197f96d640 1 -- 172.21.15.16:0/780357230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19780b4b30 con 0x7f19780cb930 2024-09-12T01:33:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.271+0000 7f196dffb640 1 -- 172.21.15.16:0/780357230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974004c10 con 0x7f19780cb930 2024-09-12T01:33:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.271+0000 7f197f96d640 1 -- 172.21.15.16:0/780357230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19780b50f0 con 0x7f19780cb930 2024-09-12T01:33:49.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.272+0000 7f197f96d640 1 -- 172.21.15.16:0/780357230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19780cc150 con 0x7f19780cb930 2024-09-12T01:33:49.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.273+0000 7f196dffb640 1 -- 172.21.15.16:0/780357230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1974011070 con 0x7f19780cb930 2024-09-12T01:33:49.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.273+0000 7f196dffb640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f195c06be60 0x7f195c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.273+0000 7f196dffb640 1 -- 172.21.15.16:0/780357230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f197408ad80 con 0x7f19780cb930 2024-09-12T01:33:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.273+0000 7f197cf08640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f195c06be60 0x7f195c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.274+0000 7f197cf08640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f195c06be60 0x7f195c06e320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1968034000 tx=0x7f196802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:49.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.277+0000 7f196dffb640 1 -- 172.21.15.16:0/780357230 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f197401a2b0 con 0x7f19780cb930 2024-09-12T01:33:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:49.519+0000 7f197f96d640 1 -- 172.21.15.16:0/780357230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]} v 0) -- 0x7f19780b5f20 con 0x7f19780cb930 2024-09-12T01:33:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.315+0000 7f196dffb640 1 -- 172.21.15.16:0/780357230 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]}]=0 v59) ==== 141+0+0 (secure 0 0 0) 0x7f19740593f0 con 0x7f19780cb930 2024-09-12T01:33:50.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.318+0000 7f197f96d640 1 -- 172.21.15.16:0/780357230 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f195c06be60 msgr2=0x7f195c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.318+0000 7f197f96d640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f195c06be60 0x7f195c06e320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1968034000 tx=0x7f196802e000 comp rx=0 tx=0).stop 2024-09-12T01:33:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.318+0000 7f197f96d640 1 -- 172.21.15.16:0/780357230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19780cb930 msgr2=0x7f19780b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.318+0000 7f197f96d640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19780cb930 0x7f19780b0bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f197400c3f0 tx=0x7f197400c8b0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.318+0000 7f197f96d640 1 -- 172.21.15.16:0/780357230 shutdown_connections 2024-09-12T01:33:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.318+0000 7f197f96d640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f195c06be60 0x7f195c06e320 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.318+0000 7f197f96d640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f197815c2b0 0x7f197809c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.318+0000 7f197f96d640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19780cc540 0x7f19780b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.318+0000 7f197f96d640 1 --2- 172.21.15.16:0/780357230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19780cb930 0x7f19780b0bf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.318+0000 7f197f96d640 1 -- 172.21.15.16:0/780357230 >> 172.21.15.16:0/780357230 conn(0x7f19780cffa0 msgr2=0x7f19780ca5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.318+0000 7f197f96d640 1 -- 172.21.15.16:0/780357230 shutdown_connections 2024-09-12T01:33:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.318+0000 7f197f96d640 1 -- 172.21.15.16:0/780357230 wait complete. 2024-09-12T01:33:50.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T01:33:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.444+0000 7f4034242640 1 Processor -- start 2024-09-12T01:33:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.444+0000 7f4034242640 1 -- start start 2024-09-12T01:33:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.445+0000 7f4034242640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f402c0d0890 0x7f402c0af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.445+0000 7f4034242640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f402c0ce480 con 0x7f402c0cddc0 2024-09-12T01:33:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.445+0000 7f4034242640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f402c0ce760 con 0x7f402c15af20 2024-09-12T01:33:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.445+0000 7f4034242640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f402c0cf090 con 0x7f402c0d0890 2024-09-12T01:33:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.446+0000 7f40327df640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f402c0d0890 0x7f402c0af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.446+0000 7f40327df640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f402c0d0890 0x7f402c0af600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46000/0 (socket says 172.21.15.16:46000) 2024-09-12T01:33:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.446+0000 7f40327df640 1 -- 172.21.15.16:0/63028569 learned_addr learned my addr 172.21.15.16:0/63028569 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.446+0000 7f40327df640 1 -- 172.21.15.16:0/63028569 >> v1:172.21.15.33:6789/0 conn(0x7f402c15af20 legacy=0x7f402c0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.446+0000 7f40327df640 1 -- 172.21.15.16:0/63028569 >> v1:172.21.15.16:6789/0 conn(0x7f402c0cddc0 legacy=0x7f402c0d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.446+0000 7f40327df640 1 -- 172.21.15.16:0/63028569 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c0cea40 con 0x7f402c0d0890 2024-09-12T01:33:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.446+0000 7f4030fdc640 1 -- 172.21.15.16:0/63028569 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3645399731 0 0) 0x7f402c0ce480 con 0x7f402c0cddc0 2024-09-12T01:33:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.447+0000 7f40327df640 1 --2- 172.21.15.16:0/63028569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402c0d0890 0x7f402c0af600 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f402c0cf210 tx=0x7f402402a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cb9697e8ee302 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.447+0000 7f4030fdc640 1 -- 172.21.15.16:0/63028569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024036070 con 0x7f402c0d0890 2024-09-12T01:33:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.447+0000 7f4030fdc640 1 -- 172.21.15.16:0/63028569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4024031600 con 0x7f402c0d0890 2024-09-12T01:33:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.447+0000 7f4030fdc640 1 -- 172.21.15.16:0/63028569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40240319d0 con 0x7f402c0d0890 2024-09-12T01:33:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.448+0000 7f4034242640 1 -- 172.21.15.16:0/63028569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402c0d0890 msgr2=0x7f402c0af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.448+0000 7f4034242640 1 --2- 172.21.15.16:0/63028569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402c0d0890 0x7f402c0af600 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f402c0cf210 tx=0x7f402402a750 comp rx=0 tx=0).stop 2024-09-12T01:33:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.448+0000 7f4034242640 1 -- 172.21.15.16:0/63028569 shutdown_connections 2024-09-12T01:33:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.448+0000 7f4034242640 1 --2- 172.21.15.16:0/63028569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402c0d0890 0x7f402c0af600 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.448+0000 7f4034242640 1 -- 172.21.15.16:0/63028569 >> 172.21.15.16:0/63028569 conn(0x7f402c06baf0 msgr2=0x7f402c06bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.448+0000 7f4034242640 1 -- 172.21.15.16:0/63028569 shutdown_connections 2024-09-12T01:33:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.448+0000 7f4034242640 1 -- 172.21.15.16:0/63028569 wait complete. 2024-09-12T01:33:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.449+0000 7f4034242640 1 Processor -- start 2024-09-12T01:33:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.449+0000 7f4034242640 1 -- start start 2024-09-12T01:33:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f4034242640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f402c0cddc0 0x7f402c15a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f4034242640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402c0d0890 0x7f402c0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f4034242640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f402c15af20 0x7f402c0c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f4034242640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f402c15b300 con 0x7f402c0d0890 2024-09-12T01:33:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f4034242640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f402c0ce230 con 0x7f402c15af20 2024-09-12T01:33:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f4034242640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f402c15b480 con 0x7f402c0cddc0 2024-09-12T01:33:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f40317dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402c0d0890 0x7f402c0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f40317dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402c0d0890 0x7f402c0c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46006/0 (socket says 172.21.15.16:46006) 2024-09-12T01:33:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f40317dd640 1 -- 172.21.15.16:0/3025182285 learned_addr learned my addr 172.21.15.16:0/3025182285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f40327df640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f402c15af20 0x7f402c0c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f40317dd640 1 -- 172.21.15.16:0/3025182285 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f402c0cddc0 msgr2=0x7f402c15a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f4031fde640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f402c0cddc0 0x7f402c15a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f40317dd640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f402c0cddc0 0x7f402c15a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f40317dd640 1 -- 172.21.15.16:0/3025182285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f402c15af20 msgr2=0x7f402c0c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f40317dd640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f402c15af20 0x7f402c0c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f40317dd640 1 -- 172.21.15.16:0/3025182285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c0ca550 con 0x7f402c0d0890 2024-09-12T01:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f4031fde640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f402c0cddc0 0x7f402c15a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.450+0000 7f40327df640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f402c15af20 0x7f402c0c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.451+0000 7f40317dd640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402c0d0890 0x7f402c0c99b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f402000c4e0 tx=0x7f402000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.451+0000 7f401e7fc640 1 -- 172.21.15.16:0/3025182285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020017070 con 0x7f402c0d0890 2024-09-12T01:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.451+0000 7f401e7fc640 1 -- 172.21.15.16:0/3025182285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40200035f0 con 0x7f402c0d0890 2024-09-12T01:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.451+0000 7f4034242640 1 -- 172.21.15.16:0/3025182285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c1f5220 con 0x7f402c0d0890 2024-09-12T01:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.451+0000 7f401e7fc640 1 -- 172.21.15.16:0/3025182285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40200039e0 con 0x7f402c0d0890 2024-09-12T01:33:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.451+0000 7f4034242640 1 -- 172.21.15.16:0/3025182285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c1f57e0 con 0x7f402c0d0890 2024-09-12T01:33:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.452+0000 7f4034242640 1 -- 172.21.15.16:0/3025182285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402c0af600 con 0x7f402c0d0890 2024-09-12T01:33:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.453+0000 7f401e7fc640 1 -- 172.21.15.16:0/3025182285 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f402001a070 con 0x7f402c0d0890 2024-09-12T01:33:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.453+0000 7f401e7fc640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f401006be30 0x7f401006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.453+0000 7f401e7fc640 1 -- 172.21.15.16:0/3025182285 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f402008aba0 con 0x7f402c0d0890 2024-09-12T01:33:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.453+0000 7f4031fde640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f401006be30 0x7f401006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.454+0000 7f4031fde640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f401006be30 0x7f401006e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f402800a830 tx=0x7f4028008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:50.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.457+0000 7f401e7fc640 1 -- 172.21.15.16:0/3025182285 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4020058d30 con 0x7f402c0d0890 2024-09-12T01:33:50.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.635+0000 7f4034242640 1 -- 172.21.15.16:0/3025182285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f402c0ae280 con 0x7f402c0d0890 2024-09-12T01:33:50.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.638+0000 7f401e7fc640 1 -- 172.21.15.16:0/3025182285 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f402005d030 con 0x7f402c0d0890 2024-09-12T01:33:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.641+0000 7f4034242640 1 -- 172.21.15.16:0/3025182285 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f401006be30 msgr2=0x7f401006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.641+0000 7f4034242640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f401006be30 0x7f401006e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f402800a830 tx=0x7f4028008040 comp rx=0 tx=0).stop 2024-09-12T01:33:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.641+0000 7f4034242640 1 -- 172.21.15.16:0/3025182285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402c0d0890 msgr2=0x7f402c0c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.641+0000 7f4034242640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402c0d0890 0x7f402c0c99b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f402000c4e0 tx=0x7f402000c9a0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.641+0000 7f4034242640 1 -- 172.21.15.16:0/3025182285 shutdown_connections 2024-09-12T01:33:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.641+0000 7f4034242640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f401006be30 0x7f401006e2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.641+0000 7f4034242640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f402c15af20 0x7f402c0c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.641+0000 7f4034242640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402c0d0890 0x7f402c0c99b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.641+0000 7f4034242640 1 --2- 172.21.15.16:0/3025182285 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f402c0cddc0 0x7f402c15a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.641+0000 7f4034242640 1 -- 172.21.15.16:0/3025182285 >> 172.21.15.16:0/3025182285 conn(0x7f402c06baf0 msgr2=0x7f402c06bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.641+0000 7f4034242640 1 -- 172.21.15.16:0/3025182285 shutdown_connections 2024-09-12T01:33:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.641+0000 7f4034242640 1 -- 172.21.15.16:0/3025182285 wait complete. 2024-09-12T01:33:50.653 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T01:33:50.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T01:33:50.700 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T01:33:50.743 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T01:33:50.743 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:33:50.743 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:50.760 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:33:50.760 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:50.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T01:33:50.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.963+0000 7f254e5b9640 1 Processor -- start 2024-09-12T01:33:50.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.963+0000 7f254e5b9640 1 -- start start 2024-09-12T01:33:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.964+0000 7f254e5b9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f254815e640 0x7f254806baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.964+0000 7f254e5b9640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f25480d0a50 0x7f25480af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.964+0000 7f254e5b9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25480730c0 con 0x7f254806c030 2024-09-12T01:33:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.964+0000 7f254e5b9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f254806c4a0 con 0x7f254815e640 2024-09-12T01:33:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.964+0000 7f254e5b9640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f25480adfa0 con 0x7f25480d0a50 2024-09-12T01:33:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.964+0000 7f2547fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f254815e640 0x7f254806baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.964+0000 7f253ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f254806c030 0x7f25480d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43900/0 (socket says 172.21.15.16:43900) 2024-09-12T01:33:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.964+0000 7f253ffff640 1 -- 172.21.15.16:0/97160881 learned_addr learned my addr 172.21.15.16:0/97160881 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.965+0000 7f254cb56640 1 --2- 172.21.15.16:0/97160881 >> v2:172.21.15.33:3300/0 conn(0x7f25480d0a50 0x7f25480af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.965+0000 7f2547fff640 1 -- 172.21.15.16:0/97160881 >> v2:172.21.15.33:3300/0 conn(0x7f25480d0a50 msgr2=0x7f25480af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.965+0000 7f2547fff640 1 --2- 172.21.15.16:0/97160881 >> v2:172.21.15.33:3300/0 conn(0x7f25480d0a50 0x7f25480af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.965+0000 7f2547fff640 1 -- 172.21.15.16:0/97160881 >> v1:172.21.15.16:6789/0 conn(0x7f254806c030 legacy=0x7f25480d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.965+0000 7f2547fff640 1 -- 172.21.15.16:0/97160881 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25480ae280 con 0x7f254815e640 2024-09-12T01:33:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.965+0000 7f25477fe640 1 -- 172.21.15.16:0/97160881 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2821950514 0 0) 0x7f2530001110 con 0x7f254806c030 2024-09-12T01:33:50.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.965+0000 7f2547fff640 1 --2- 172.21.15.16:0/97160881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f254815e640 0x7f254806baf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f25480b71a0 tx=0x7f253802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cff5717eb61ca84 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:50.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.966+0000 7f25477fe640 1 -- 172.21.15.16:0/97160881 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538037070 con 0x7f254815e640 2024-09-12T01:33:50.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.966+0000 7f25477fe640 1 -- 172.21.15.16:0/97160881 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253802add0 con 0x7f254815e640 2024-09-12T01:33:50.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.966+0000 7f25477fe640 1 -- 172.21.15.16:0/97160881 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538032660 con 0x7f254815e640 2024-09-12T01:33:50.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.966+0000 7f254e5b9640 1 -- 172.21.15.16:0/97160881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f254815e640 msgr2=0x7f254806baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.966+0000 7f254e5b9640 1 --2- 172.21.15.16:0/97160881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f254815e640 0x7f254806baf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f25480b71a0 tx=0x7f253802a080 comp rx=0 tx=0).stop 2024-09-12T01:33:50.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.966+0000 7f254e5b9640 1 -- 172.21.15.16:0/97160881 shutdown_connections 2024-09-12T01:33:50.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.966+0000 7f254e5b9640 1 --2- 172.21.15.16:0/97160881 >> v2:172.21.15.33:3300/0 conn(0x7f25480d0a50 0x7f25480af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.966+0000 7f254e5b9640 1 --2- 172.21.15.16:0/97160881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f254815e640 0x7f254806baf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.966+0000 7f254e5b9640 1 -- 172.21.15.16:0/97160881 >> 172.21.15.16:0/97160881 conn(0x7f25480b2cc0 msgr2=0x7f25480b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:50.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.966+0000 7f254e5b9640 1 -- 172.21.15.16:0/97160881 shutdown_connections 2024-09-12T01:33:50.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.966+0000 7f254e5b9640 1 -- 172.21.15.16:0/97160881 wait complete. 2024-09-12T01:33:50.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.968+0000 7f254e5b9640 1 Processor -- start 2024-09-12T01:33:50.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.968+0000 7f254e5b9640 1 -- start start 2024-09-12T01:33:50.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.968+0000 7f254e5b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f254806c030 0x7f25480b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.968+0000 7f254e5b9640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f25480d0a50 0x7f2548158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.968+0000 7f254e5b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254815e640 0x7f254815a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.968+0000 7f254e5b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f25480af580 con 0x7f254806c030 2024-09-12T01:33:50.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.968+0000 7f254e5b9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f25480ae840 con 0x7f254815e640 2024-09-12T01:33:50.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.968+0000 7f254e5b9640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f25480af700 con 0x7f25480d0a50 2024-09-12T01:33:50.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.968+0000 7f2547fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f254806c030 0x7f25480b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f2547fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f254806c030 0x7f25480b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46024/0 (socket says 172.21.15.16:46024) 2024-09-12T01:33:50.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f2547fff640 1 -- 172.21.15.16:0/2082820981 learned_addr learned my addr 172.21.15.16:0/2082820981 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:50.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f254cb56640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254815e640 0x7f254815a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f253ffff640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f25480d0a50 0x7f2548158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f2547fff640 1 -- 172.21.15.16:0/2082820981 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f25480d0a50 msgr2=0x7f2548158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f2547fff640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f25480d0a50 0x7f2548158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f2547fff640 1 -- 172.21.15.16:0/2082820981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254815e640 msgr2=0x7f254815a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f2547fff640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254815e640 0x7f254815a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f2547fff640 1 -- 172.21.15.16:0/2082820981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25480b5350 con 0x7f254806c030 2024-09-12T01:33:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f2547fff640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f254806c030 0x7f25480b4940 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2538000c00 tx=0x7f25380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f25457fa640 1 -- 172.21.15.16:0/2082820981 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538037040 con 0x7f254806c030 2024-09-12T01:33:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f25457fa640 1 -- 172.21.15.16:0/2082820981 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2538042070 con 0x7f254806c030 2024-09-12T01:33:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f254e5b9640 1 -- 172.21.15.16:0/2082820981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25480b5630 con 0x7f254806c030 2024-09-12T01:33:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f25457fa640 1 -- 172.21.15.16:0/2082820981 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253803b890 con 0x7f254806c030 2024-09-12T01:33:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.969+0000 7f254e5b9640 1 -- 172.21.15.16:0/2082820981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25480b5bf0 con 0x7f254806c030 2024-09-12T01:33:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.970+0000 7f254e5b9640 1 -- 172.21.15.16:0/2082820981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254815ad20 con 0x7f254806c030 2024-09-12T01:33:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.971+0000 7f25457fa640 1 -- 172.21.15.16:0/2082820981 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f253802ac90 con 0x7f254806c030 2024-09-12T01:33:50.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.971+0000 7f25457fa640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f252806be30 0x7f252806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.972+0000 7f25457fa640 1 -- 172.21.15.16:0/2082820981 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f25380ab730 con 0x7f254806c030 2024-09-12T01:33:50.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.972+0000 7f253ffff640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f252806be30 0x7f252806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.972+0000 7f253ffff640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f252806be30 0x7f252806e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25480b71a0 tx=0x7f2530007970 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:50.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:50.976+0000 7f25457fa640 1 -- 172.21.15.16:0/2082820981 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2538079da0 con 0x7f254806c030 2024-09-12T01:33:51.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.166+0000 7f254e5b9640 1 -- 172.21.15.16:0/2082820981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f25480af040 con 0x7f254806c030 2024-09-12T01:33:51.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.167+0000 7f25457fa640 1 -- 172.21.15.16:0/2082820981 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f25380796a0 con 0x7f254806c030 2024-09-12T01:33:51.167 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-12T01:33:51.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.169+0000 7f254e5b9640 1 -- 172.21.15.16:0/2082820981 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f252806be30 msgr2=0x7f252806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:51.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.169+0000 7f254e5b9640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f252806be30 0x7f252806e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25480b71a0 tx=0x7f2530007970 comp rx=0 tx=0).stop 2024-09-12T01:33:51.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.169+0000 7f254e5b9640 1 -- 172.21.15.16:0/2082820981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f254806c030 msgr2=0x7f25480b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.169+0000 7f254e5b9640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f254806c030 0x7f25480b4940 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2538000c00 tx=0x7f25380046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.169+0000 7f254e5b9640 1 -- 172.21.15.16:0/2082820981 shutdown_connections 2024-09-12T01:33:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.169+0000 7f254e5b9640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f252806be30 0x7f252806e2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.169+0000 7f254e5b9640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254815e640 0x7f254815a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.169+0000 7f254e5b9640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f25480d0a50 0x7f2548158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.169+0000 7f254e5b9640 1 --2- 172.21.15.16:0/2082820981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f254806c030 0x7f25480b4940 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.169+0000 7f254e5b9640 1 -- 172.21.15.16:0/2082820981 >> 172.21.15.16:0/2082820981 conn(0x7f25480b2cc0 msgr2=0x7f25480b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:51.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.169+0000 7f254e5b9640 1 -- 172.21.15.16:0/2082820981 shutdown_connections 2024-09-12T01:33:51.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.169+0000 7f254e5b9640 1 -- 172.21.15.16:0/2082820981 wait complete. 2024-09-12T01:33:51.180 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:33:51.180 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:51.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T01:33:51.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.342+0000 7f003e54b640 1 Processor -- start 2024-09-12T01:33:51.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.342+0000 7f003e54b640 1 -- start start 2024-09-12T01:33:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.342+0000 7f003e54b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f003815e8d0 0x7f0038162e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.342+0000 7f003e54b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f003806baf0 0x7f003806bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.342+0000 7f003e54b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00380cb790 con 0x7f00380ced10 2024-09-12T01:33:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.342+0000 7f003e54b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f00380ce8d0 con 0x7f003815e8d0 2024-09-12T01:33:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.342+0000 7f003e54b640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f003806c470 con 0x7f003806baf0 2024-09-12T01:33:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.343+0000 7f0037fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f003815e8d0 0x7f0038162e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.343+0000 7f003cae8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f00380ced10 0x7f00380cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43906/0 (socket says 172.21.15.16:43906) 2024-09-12T01:33:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.343+0000 7f003cae8640 1 -- 172.21.15.16:0/4094319854 learned_addr learned my addr 172.21.15.16:0/4094319854 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.343+0000 7f00377fe640 1 --2- 172.21.15.16:0/4094319854 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f003806baf0 0x7f003806bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.343+0000 7f0037fff640 1 -- 172.21.15.16:0/4094319854 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f003806baf0 msgr2=0x7f003806bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.343+0000 7f0037fff640 1 --2- 172.21.15.16:0/4094319854 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f003806baf0 0x7f003806bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.343+0000 7f0037fff640 1 -- 172.21.15.16:0/4094319854 >> v1:172.21.15.16:6789/0 conn(0x7f00380ced10 legacy=0x7f00380cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.343+0000 7f0037fff640 1 -- 172.21.15.16:0/4094319854 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00380cda20 con 0x7f003815e8d0 2024-09-12T01:33:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.343+0000 7f0036ffd640 1 -- 172.21.15.16:0/4094319854 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1732710734 0 0) 0x7f00380cb790 con 0x7f00380ced10 2024-09-12T01:33:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.343+0000 7f00377fe640 1 --2- 172.21.15.16:0/4094319854 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f003806baf0 0x7f003806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.344+0000 7f0037fff640 1 --2- 172.21.15.16:0/4094319854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003815e8d0 0x7f0038162e70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f00280097a0 tx=0x7f002802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8752cb19fbdf59de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.344+0000 7f0036ffd640 1 -- 172.21.15.16:0/4094319854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028037070 con 0x7f003815e8d0 2024-09-12T01:33:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.344+0000 7f0036ffd640 1 -- 172.21.15.16:0/4094319854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002802ae50 con 0x7f003815e8d0 2024-09-12T01:33:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.344+0000 7f0036ffd640 1 -- 172.21.15.16:0/4094319854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028032660 con 0x7f003815e8d0 2024-09-12T01:33:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.344+0000 7f003e54b640 1 -- 172.21.15.16:0/4094319854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003815e8d0 msgr2=0x7f0038162e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.344+0000 7f003e54b640 1 --2- 172.21.15.16:0/4094319854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003815e8d0 0x7f0038162e70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f00280097a0 tx=0x7f002802a100 comp rx=0 tx=0).stop 2024-09-12T01:33:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.345+0000 7f003e54b640 1 -- 172.21.15.16:0/4094319854 shutdown_connections 2024-09-12T01:33:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.345+0000 7f003e54b640 1 --2- 172.21.15.16:0/4094319854 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f003806baf0 0x7f003806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.345+0000 7f003e54b640 1 --2- 172.21.15.16:0/4094319854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003815e8d0 0x7f0038162e70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.345+0000 7f003e54b640 1 -- 172.21.15.16:0/4094319854 >> 172.21.15.16:0/4094319854 conn(0x7f00380b2d10 msgr2=0x7f00380b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.345+0000 7f003e54b640 1 -- 172.21.15.16:0/4094319854 shutdown_connections 2024-09-12T01:33:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.345+0000 7f003e54b640 1 -- 172.21.15.16:0/4094319854 wait complete. 2024-09-12T01:33:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.346+0000 7f003e54b640 1 Processor -- start 2024-09-12T01:33:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.346+0000 7f003e54b640 1 -- start start 2024-09-12T01:33:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003e54b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f003806baf0 0x7f003815c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003e54b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00380ced10 0x7f003815cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003e54b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003815e8d0 0x7f00380cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003e54b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00380cdfa0 con 0x7f003815e8d0 2024-09-12T01:33:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003e54b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00381633b0 con 0x7f00380ced10 2024-09-12T01:33:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003e54b640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f00380ce280 con 0x7f003806baf0 2024-09-12T01:33:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003cae8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003815e8d0 0x7f00380cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003cae8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003815e8d0 0x7f00380cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46036/0 (socket says 172.21.15.16:46036) 2024-09-12T01:33:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003cae8640 1 -- 172.21.15.16:0/824478111 learned_addr learned my addr 172.21.15.16:0/824478111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f0037fff640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f003806baf0 0x7f003815c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f00377fe640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00380ced10 0x7f003815cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003cae8640 1 -- 172.21.15.16:0/824478111 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f003806baf0 msgr2=0x7f003815c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003cae8640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f003806baf0 0x7f003815c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003cae8640 1 -- 172.21.15.16:0/824478111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00380ced10 msgr2=0x7f003815cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003cae8640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00380ced10 0x7f003815cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003cae8640 1 -- 172.21.15.16:0/824478111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003815d550 con 0x7f003815e8d0 2024-09-12T01:33:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.347+0000 7f003cae8640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003815e8d0 0x7f00380cba80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f003000c800 tx=0x7f003000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.348+0000 7f0034ff9640 1 -- 172.21.15.16:0/824478111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030018070 con 0x7f003815e8d0 2024-09-12T01:33:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.348+0000 7f0034ff9640 1 -- 172.21.15.16:0/824478111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0030004d80 con 0x7f003815e8d0 2024-09-12T01:33:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.348+0000 7f003e54b640 1 -- 172.21.15.16:0/824478111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00381f44b0 con 0x7f003815e8d0 2024-09-12T01:33:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.348+0000 7f0034ff9640 1 -- 172.21.15.16:0/824478111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003000a880 con 0x7f003815e8d0 2024-09-12T01:33:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.348+0000 7f003e54b640 1 -- 172.21.15.16:0/824478111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003815c8a0 con 0x7f003815e8d0 2024-09-12T01:33:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.349+0000 7f003e54b640 1 -- 172.21.15.16:0/824478111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00380ce560 con 0x7f003815e8d0 2024-09-12T01:33:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.349+0000 7f0034ff9640 1 -- 172.21.15.16:0/824478111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f003001b070 con 0x7f003815e8d0 2024-09-12T01:33:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.350+0000 7f0034ff9640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f002406be30 0x7f002406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.350+0000 7f0034ff9640 1 -- 172.21.15.16:0/824478111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f003008a6c0 con 0x7f003815e8d0 2024-09-12T01:33:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.350+0000 7f0037fff640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f002406be30 0x7f002406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.350+0000 7f0037fff640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f002406be30 0x7f002406e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0028034000 tx=0x7f002802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:51.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.354+0000 7f0034ff9640 1 -- 172.21.15.16:0/824478111 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0030058e80 con 0x7f003815e8d0 2024-09-12T01:33:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.543+0000 7f003e54b640 1 -- 172.21.15.16:0/824478111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f003806eca0 con 0x7f003815e8d0 2024-09-12T01:33:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.543+0000 7f0034ff9640 1 -- 172.21.15.16:0/824478111 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f0030058780 con 0x7f003815e8d0 2024-09-12T01:33:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-12T01:33:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.546+0000 7f003e54b640 1 -- 172.21.15.16:0/824478111 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f002406be30 msgr2=0x7f002406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.546+0000 7f003e54b640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f002406be30 0x7f002406e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0028034000 tx=0x7f002802e000 comp rx=0 tx=0).stop 2024-09-12T01:33:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.546+0000 7f003e54b640 1 -- 172.21.15.16:0/824478111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003815e8d0 msgr2=0x7f00380cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.546+0000 7f003e54b640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003815e8d0 0x7f00380cba80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f003000c800 tx=0x7f003000ccc0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.546+0000 7f003e54b640 1 -- 172.21.15.16:0/824478111 shutdown_connections 2024-09-12T01:33:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.546+0000 7f003e54b640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f002406be30 0x7f002406e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.546+0000 7f003e54b640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003815e8d0 0x7f00380cba80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.546+0000 7f003e54b640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00380ced10 0x7f003815cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.546+0000 7f003e54b640 1 --2- 172.21.15.16:0/824478111 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f003806baf0 0x7f003815c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.546+0000 7f003e54b640 1 -- 172.21.15.16:0/824478111 >> 172.21.15.16:0/824478111 conn(0x7f00380b2d10 msgr2=0x7f00380af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.546+0000 7f003e54b640 1 -- 172.21.15.16:0/824478111 shutdown_connections 2024-09-12T01:33:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.546+0000 7f003e54b640 1 -- 172.21.15.16:0/824478111 wait complete. 2024-09-12T01:33:51.556 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-12T01:33:51.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:51.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-12T01:33:51.585 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:33:51.585 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:33:51.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:51.585 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T01:33:51.642 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:33:51.642 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:51.663 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:33:51.663 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:51.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T01:33:51.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.828+0000 7fc1432a5640 1 Processor -- start 2024-09-12T01:33:51.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.828+0000 7fc1432a5640 1 -- start start 2024-09-12T01:33:51.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc1432a5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc13c15e8d0 0x7fc13c06dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:51.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc1432a5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc13c0acdd0 0x7fc13c0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:51.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc1432a5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc13c06f530 con 0x7fc13c0cb3b0 2024-09-12T01:33:51.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc1432a5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc13c0cbbb0 con 0x7fc13c0acdd0 2024-09-12T01:33:51.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc1432a5640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fc13c0cbe90 con 0x7fc13c15e8d0 2024-09-12T01:33:51.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc140840640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc13c0acdd0 0x7fc13c0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:51.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc141842640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc13c0cb3b0 0x7fc13c0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43910/0 (socket says 172.21.15.16:43910) 2024-09-12T01:33:51.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc141842640 1 -- 172.21.15.16:0/812343524 learned_addr learned my addr 172.21.15.16:0/812343524 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:51.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc141041640 1 --2- 172.21.15.16:0/812343524 >> v2:172.21.15.33:3300/0 conn(0x7fc13c15e8d0 0x7fc13c06dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:51.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc140840640 1 -- 172.21.15.16:0/812343524 >> v2:172.21.15.33:3300/0 conn(0x7fc13c15e8d0 msgr2=0x7fc13c06dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:51.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc140840640 1 --2- 172.21.15.16:0/812343524 >> v2:172.21.15.33:3300/0 conn(0x7fc13c15e8d0 0x7fc13c06dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc140840640 1 -- 172.21.15.16:0/812343524 >> v1:172.21.15.16:6789/0 conn(0x7fc13c0cb3b0 legacy=0x7fc13c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:51.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc12ffff640 1 -- 172.21.15.16:0/812343524 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1988231031 0 0) 0x7fc13c06f530 con 0x7fc13c0cb3b0 2024-09-12T01:33:51.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.829+0000 7fc140840640 1 -- 172.21.15.16:0/812343524 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13c0ca5f0 con 0x7fc13c0acdd0 2024-09-12T01:33:51.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.830+0000 7fc140840640 1 --2- 172.21.15.16:0/812343524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c0acdd0 0x7fc13c0cae70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc1380097a0 tx=0x7fc13802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6c72f4942cfbb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:51.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.830+0000 7fc12ffff640 1 -- 172.21.15.16:0/812343524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138037070 con 0x7fc13c0acdd0 2024-09-12T01:33:51.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.830+0000 7fc12ffff640 1 -- 172.21.15.16:0/812343524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc138032440 con 0x7fc13c0acdd0 2024-09-12T01:33:51.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.830+0000 7fc12ffff640 1 -- 172.21.15.16:0/812343524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138032810 con 0x7fc13c0acdd0 2024-09-12T01:33:51.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.831+0000 7fc1432a5640 1 -- 172.21.15.16:0/812343524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c0acdd0 msgr2=0x7fc13c0cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:51.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.831+0000 7fc1432a5640 1 --2- 172.21.15.16:0/812343524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c0acdd0 0x7fc13c0cae70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc1380097a0 tx=0x7fc13802a1e0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.831+0000 7fc1432a5640 1 -- 172.21.15.16:0/812343524 shutdown_connections 2024-09-12T01:33:51.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.831+0000 7fc1432a5640 1 --2- 172.21.15.16:0/812343524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c0acdd0 0x7fc13c0cae70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.831+0000 7fc1432a5640 1 --2- 172.21.15.16:0/812343524 >> v2:172.21.15.33:3300/0 conn(0x7fc13c15e8d0 0x7fc13c06dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.831+0000 7fc1432a5640 1 -- 172.21.15.16:0/812343524 >> 172.21.15.16:0/812343524 conn(0x7fc13c0cffa0 msgr2=0x7fc13c0d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:51.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.831+0000 7fc1432a5640 1 -- 172.21.15.16:0/812343524 shutdown_connections 2024-09-12T01:33:51.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.831+0000 7fc1432a5640 1 -- 172.21.15.16:0/812343524 wait complete. 2024-09-12T01:33:51.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.832+0000 7fc1432a5640 1 Processor -- start 2024-09-12T01:33:51.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.832+0000 7fc1432a5640 1 -- start start 2024-09-12T01:33:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc1432a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c0acdd0 0x7fc13c14b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc1432a5640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc13c0cb3b0 0x7fc13c0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc1432a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc13c15e8d0 0x7fc13c09cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc1432a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c0cb840 con 0x7fc13c0acdd0 2024-09-12T01:33:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc1432a5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c0cabb0 con 0x7fc13c15e8d0 2024-09-12T01:33:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc1432a5640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fc13c073720 con 0x7fc13c0cb3b0 2024-09-12T01:33:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc141041640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c0acdd0 0x7fc13c14b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc141041640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c0acdd0 0x7fc13c14b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46046/0 (socket says 172.21.15.16:46046) 2024-09-12T01:33:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc141041640 1 -- 172.21.15.16:0/1447099179 learned_addr learned my addr 172.21.15.16:0/1447099179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc141842640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc13c15e8d0 0x7fc13c09cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc140840640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc13c0cb3b0 0x7fc13c0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc141041640 1 -- 172.21.15.16:0/1447099179 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc13c0cb3b0 msgr2=0x7fc13c0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc141041640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc13c0cb3b0 0x7fc13c0b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc141041640 1 -- 172.21.15.16:0/1447099179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc13c15e8d0 msgr2=0x7fc13c09cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc141041640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc13c15e8d0 0x7fc13c09cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.833+0000 7fc141041640 1 -- 172.21.15.16:0/1447099179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13c0b4900 con 0x7fc13c0acdd0 2024-09-12T01:33:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.834+0000 7fc141041640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c0acdd0 0x7fc13c14b340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc13000c820 tx=0x7fc13000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.834+0000 7fc12dffb640 1 -- 172.21.15.16:0/1447099179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130015070 con 0x7fc13c0acdd0 2024-09-12T01:33:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.834+0000 7fc12dffb640 1 -- 172.21.15.16:0/1447099179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc130004020 con 0x7fc13c0acdd0 2024-09-12T01:33:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.834+0000 7fc1432a5640 1 -- 172.21.15.16:0/1447099179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13c0b3280 con 0x7fc13c0acdd0 2024-09-12T01:33:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.834+0000 7fc12dffb640 1 -- 172.21.15.16:0/1447099179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1300043f0 con 0x7fc13c0acdd0 2024-09-12T01:33:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.834+0000 7fc1432a5640 1 -- 172.21.15.16:0/1447099179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc13c0b3840 con 0x7fc13c0acdd0 2024-09-12T01:33:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.835+0000 7fc1432a5640 1 -- 172.21.15.16:0/1447099179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc13c073ab0 con 0x7fc13c0acdd0 2024-09-12T01:33:51.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.835+0000 7fc12dffb640 1 -- 172.21.15.16:0/1447099179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc130007830 con 0x7fc13c0acdd0 2024-09-12T01:33:51.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.836+0000 7fc12dffb640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc12006be30 0x7fc12006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:51.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.836+0000 7fc140840640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc12006be30 0x7fc12006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:51.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.836+0000 7fc12dffb640 1 -- 172.21.15.16:0/1447099179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc13008afd0 con 0x7fc13c0acdd0 2024-09-12T01:33:51.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.836+0000 7fc140840640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc12006be30 0x7fc12006e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc138034000 tx=0x7fc13802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:51.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:51.840+0000 7fc12dffb640 1 -- 172.21.15.16:0/1447099179 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc13c0b4900 con 0x7fc13c0acdd0 2024-09-12T01:33:52.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.029+0000 7fc1432a5640 1 -- 172.21.15.16:0/1447099179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc13c15de10 con 0x7fc13c0acdd0 2024-09-12T01:33:52.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.030+0000 7fc12dffb640 1 -- 172.21.15.16:0/1447099179 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fc130059640 con 0x7fc13c0acdd0 2024-09-12T01:33:52.029 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-12T01:33:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.032+0000 7fc1432a5640 1 -- 172.21.15.16:0/1447099179 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc12006be30 msgr2=0x7fc12006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.032+0000 7fc1432a5640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc12006be30 0x7fc12006e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc138034000 tx=0x7fc13802e000 comp rx=0 tx=0).stop 2024-09-12T01:33:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.032+0000 7fc1432a5640 1 -- 172.21.15.16:0/1447099179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c0acdd0 msgr2=0x7fc13c14b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.032+0000 7fc1432a5640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c0acdd0 0x7fc13c14b340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc13000c820 tx=0x7fc13000cce0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.032+0000 7fc1432a5640 1 -- 172.21.15.16:0/1447099179 shutdown_connections 2024-09-12T01:33:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.032+0000 7fc1432a5640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc12006be30 0x7fc12006e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.032+0000 7fc1432a5640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc13c15e8d0 0x7fc13c09cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.032+0000 7fc1432a5640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc13c0cb3b0 0x7fc13c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.032+0000 7fc1432a5640 1 --2- 172.21.15.16:0/1447099179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c0acdd0 0x7fc13c14b340 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.032+0000 7fc1432a5640 1 -- 172.21.15.16:0/1447099179 >> 172.21.15.16:0/1447099179 conn(0x7fc13c0cffa0 msgr2=0x7fc13c0cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.032+0000 7fc1432a5640 1 -- 172.21.15.16:0/1447099179 shutdown_connections 2024-09-12T01:33:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.032+0000 7fc1432a5640 1 -- 172.21.15.16:0/1447099179 wait complete. 2024-09-12T01:33:52.042 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:33:52.042 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:52.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T01:33:52.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.166+0000 7f54d935f640 1 Processor -- start 2024-09-12T01:33:52.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.166+0000 7f54d935f640 1 -- start start 2024-09-12T01:33:52.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.167+0000 7f54d935f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54d415ffe0 0x7f54d415d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:52.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.167+0000 7f54d935f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f54d40d0210 0x7f54d40cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:52.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.167+0000 7f54d935f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d41604a0 con 0x7f54d415c130 2024-09-12T01:33:52.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.167+0000 7f54d935f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f54d4160780 con 0x7f54d415ffe0 2024-09-12T01:33:52.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.167+0000 7f54d935f640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f54d4160a60 con 0x7f54d40d0210 2024-09-12T01:33:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.167+0000 7f54d27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54d415ffe0 0x7f54d415d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.167+0000 7f54d2ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f54d415c130 0x7f54d40d1a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43920/0 (socket says 172.21.15.16:43920) 2024-09-12T01:33:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.167+0000 7f54d27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54d415ffe0 0x7f54d415d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46052/0 (socket says 172.21.15.16:46052) 2024-09-12T01:33:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.167+0000 7f54d2ffd640 1 -- 172.21.15.16:0/3756403224 learned_addr learned my addr 172.21.15.16:0/3756403224 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.168+0000 7f54d37fe640 1 --2- 172.21.15.16:0/3756403224 >> v2:172.21.15.33:3300/0 conn(0x7f54d40d0210 0x7f54d40cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.168+0000 7f54d27fc640 1 -- 172.21.15.16:0/3756403224 >> v2:172.21.15.33:3300/0 conn(0x7f54d40d0210 msgr2=0x7f54d40cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.168+0000 7f54d27fc640 1 --2- 172.21.15.16:0/3756403224 >> v2:172.21.15.33:3300/0 conn(0x7f54d40d0210 0x7f54d40cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.168+0000 7f54d27fc640 1 -- 172.21.15.16:0/3756403224 >> v1:172.21.15.16:6789/0 conn(0x7f54d415c130 legacy=0x7f54d40d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.168+0000 7f54d27fc640 1 -- 172.21.15.16:0/3756403224 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d4160d40 con 0x7f54d415ffe0 2024-09-12T01:33:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.168+0000 7f54d27fc640 1 --2- 172.21.15.16:0/3756403224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d415ffe0 0x7f54d415d520 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f54d40ce170 tx=0x7f54bc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff3c21698c95915 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.169+0000 7f54d1ffb640 1 -- 172.21.15.16:0/3756403224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc037070 con 0x7f54d415ffe0 2024-09-12T01:33:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.169+0000 7f54d1ffb640 1 -- 172.21.15.16:0/3756403224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54bc032440 con 0x7f54d415ffe0 2024-09-12T01:33:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.169+0000 7f54d1ffb640 1 -- 172.21.15.16:0/3756403224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc032810 con 0x7f54d415ffe0 2024-09-12T01:33:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.169+0000 7f54d935f640 1 -- 172.21.15.16:0/3756403224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d415ffe0 msgr2=0x7f54d415d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.169+0000 7f54d935f640 1 --2- 172.21.15.16:0/3756403224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d415ffe0 0x7f54d415d520 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f54d40ce170 tx=0x7f54bc02a1a0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.169+0000 7f54d935f640 1 -- 172.21.15.16:0/3756403224 shutdown_connections 2024-09-12T01:33:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.169+0000 7f54d935f640 1 --2- 172.21.15.16:0/3756403224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d415ffe0 0x7f54d415d520 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.169+0000 7f54d935f640 1 --2- 172.21.15.16:0/3756403224 >> v2:172.21.15.33:3300/0 conn(0x7f54d40d0210 0x7f54d40cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.169+0000 7f54d935f640 1 -- 172.21.15.16:0/3756403224 >> 172.21.15.16:0/3756403224 conn(0x7f54d406baf0 msgr2=0x7f54d406bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.169+0000 7f54d935f640 1 -- 172.21.15.16:0/3756403224 shutdown_connections 2024-09-12T01:33:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.169+0000 7f54d935f640 1 -- 172.21.15.16:0/3756403224 wait complete. 2024-09-12T01:33:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.171+0000 7f54d935f640 1 Processor -- start 2024-09-12T01:33:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.171+0000 7f54d935f640 1 -- start start 2024-09-12T01:33:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d935f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d40d0210 0x7f54d4154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d935f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f54d415c130 0x7f54d40c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d935f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d415ffe0 0x7f54d40c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d935f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54d40d0820 con 0x7f54d415ffe0 2024-09-12T01:33:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d935f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54d40d06a0 con 0x7f54d40d0210 2024-09-12T01:33:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d935f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f54d40d09a0 con 0x7f54d415c130 2024-09-12T01:33:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d415ffe0 0x7f54d40c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d415ffe0 0x7f54d40c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46064/0 (socket says 172.21.15.16:46064) 2024-09-12T01:33:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d37fe640 1 -- 172.21.15.16:0/591194559 learned_addr learned my addr 172.21.15.16:0/591194559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d2ffd640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d40d0210 0x7f54d4154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d27fc640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f54d415c130 0x7f54d40c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d37fe640 1 -- 172.21.15.16:0/591194559 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f54d415c130 msgr2=0x7f54d40c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d37fe640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f54d415c130 0x7f54d40c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d37fe640 1 -- 172.21.15.16:0/591194559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d40d0210 msgr2=0x7f54d4154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d37fe640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d40d0210 0x7f54d4154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.172+0000 7f54d37fe640 1 -- 172.21.15.16:0/591194559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d4154860 con 0x7f54d415ffe0 2024-09-12T01:33:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.173+0000 7f54d37fe640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d415ffe0 0x7f54d40c98c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f54c400d5b0 tx=0x7f54c400da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.173+0000 7f54cb7fe640 1 -- 172.21.15.16:0/591194559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c40125f0 con 0x7f54d415ffe0 2024-09-12T01:33:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.173+0000 7f54cb7fe640 1 -- 172.21.15.16:0/591194559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c4019070 con 0x7f54d415ffe0 2024-09-12T01:33:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.173+0000 7f54d935f640 1 -- 172.21.15.16:0/591194559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d40c9ed0 con 0x7f54d415ffe0 2024-09-12T01:33:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.173+0000 7f54cb7fe640 1 -- 172.21.15.16:0/591194559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c40129f0 con 0x7f54d415ffe0 2024-09-12T01:33:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.173+0000 7f54d935f640 1 -- 172.21.15.16:0/591194559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d40b5f20 con 0x7f54d415ffe0 2024-09-12T01:33:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.174+0000 7f54d935f640 1 -- 172.21.15.16:0/591194559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d4068880 con 0x7f54d415ffe0 2024-09-12T01:33:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.175+0000 7f54cb7fe640 1 -- 172.21.15.16:0/591194559 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54c4012b90 con 0x7f54d415ffe0 2024-09-12T01:33:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.175+0000 7f54cb7fe640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f54b406be30 0x7f54b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.175+0000 7f54cb7fe640 1 -- 172.21.15.16:0/591194559 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f54c408b350 con 0x7f54d415ffe0 2024-09-12T01:33:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.175+0000 7f54d2ffd640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f54b406be30 0x7f54b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.176+0000 7f54d2ffd640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f54b406be30 0x7f54b406e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0012e0 tx=0x7f54cc001a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:52.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.180+0000 7f54cb7fe640 1 -- 172.21.15.16:0/591194559 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f54d4154860 con 0x7f54d415ffe0 2024-09-12T01:33:52.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.367+0000 7f54d935f640 1 -- 172.21.15.16:0/591194559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f54d40d1a70 con 0x7f54d415ffe0 2024-09-12T01:33:52.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.367+0000 7f54cb7fe640 1 -- 172.21.15.16:0/591194559 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f54c4059400 con 0x7f54d415ffe0 2024-09-12T01:33:52.367 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-12T01:33:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.369+0000 7f54d935f640 1 -- 172.21.15.16:0/591194559 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f54b406be30 msgr2=0x7f54b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.369+0000 7f54d935f640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f54b406be30 0x7f54b406e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0012e0 tx=0x7f54cc001a30 comp rx=0 tx=0).stop 2024-09-12T01:33:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.369+0000 7f54d935f640 1 -- 172.21.15.16:0/591194559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d415ffe0 msgr2=0x7f54d40c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.369+0000 7f54d935f640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d415ffe0 0x7f54d40c98c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f54c400d5b0 tx=0x7f54c400da70 comp rx=0 tx=0).stop 2024-09-12T01:33:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.369+0000 7f54d935f640 1 -- 172.21.15.16:0/591194559 shutdown_connections 2024-09-12T01:33:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.369+0000 7f54d935f640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f54b406be30 0x7f54b406e2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.369+0000 7f54d935f640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d415ffe0 0x7f54d40c98c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.369+0000 7f54d935f640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f54d415c130 0x7f54d40c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.369+0000 7f54d935f640 1 --2- 172.21.15.16:0/591194559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d40d0210 0x7f54d4154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.369+0000 7f54d935f640 1 -- 172.21.15.16:0/591194559 >> 172.21.15.16:0/591194559 conn(0x7f54d406baf0 msgr2=0x7f54d406e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.369+0000 7f54d935f640 1 -- 172.21.15.16:0/591194559 shutdown_connections 2024-09-12T01:33:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.369+0000 7f54d935f640 1 -- 172.21.15.16:0/591194559 wait complete. 2024-09-12T01:33:52.380 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2024-09-12T01:33:52.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:52.380 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-12T01:33:52.412 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:33:52.412 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:33:52.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:52.412 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T01:33:52.478 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:33:52.478 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:52.495 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:33:52.495 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:52.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T01:33:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.695+0000 7fa247103640 1 Processor -- start 2024-09-12T01:33:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.695+0000 7fa247103640 1 -- start start 2024-09-12T01:33:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.696+0000 7fa247103640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa2400d0cc0 0x7fa2400d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.696+0000 7fa247103640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa2400b76c0 0x7fa2400b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.696+0000 7fa247103640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa240161dc0 con 0x7fa2401619e0 2024-09-12T01:33:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.696+0000 7fa247103640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa2400d3f20 con 0x7fa2400d0cc0 2024-09-12T01:33:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.696+0000 7fa247103640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fa2400d3310 con 0x7fa2400b76c0 2024-09-12T01:33:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.696+0000 7fa23ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa2400d0cc0 0x7fa2400d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.696+0000 7fa244e9f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa2401619e0 0x7fa2400d2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43930/0 (socket says 172.21.15.16:43930) 2024-09-12T01:33:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.696+0000 7fa244e9f640 1 -- 172.21.15.16:0/4168293781 learned_addr learned my addr 172.21.15.16:0/4168293781 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.697+0000 7fa2456a0640 1 --2- 172.21.15.16:0/4168293781 >> v2:172.21.15.33:3300/0 conn(0x7fa2400b76c0 0x7fa2400b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.697+0000 7fa23ffff640 1 -- 172.21.15.16:0/4168293781 >> v2:172.21.15.33:3300/0 conn(0x7fa2400b76c0 msgr2=0x7fa2400b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.697+0000 7fa23ffff640 1 --2- 172.21.15.16:0/4168293781 >> v2:172.21.15.33:3300/0 conn(0x7fa2400b76c0 0x7fa2400b7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.697+0000 7fa23ffff640 1 -- 172.21.15.16:0/4168293781 >> v1:172.21.15.16:6789/0 conn(0x7fa2401619e0 legacy=0x7fa2400d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.697+0000 7fa23ffff640 1 -- 172.21.15.16:0/4168293781 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2400d35f0 con 0x7fa2400d0cc0 2024-09-12T01:33:52.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.697+0000 7fa23f7fe640 1 -- 172.21.15.16:0/4168293781 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3973743948 0 0) 0x7fa240161dc0 con 0x7fa2401619e0 2024-09-12T01:33:52.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.697+0000 7fa2456a0640 1 --2- 172.21.15.16:0/4168293781 >> v2:172.21.15.33:3300/0 conn(0x7fa2400b76c0 0x7fa2400b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:52.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.698+0000 7fa23ffff640 1 --2- 172.21.15.16:0/4168293781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2400d0cc0 0x7fa2400d1100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa2400d40a0 tx=0x7fa23402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5a4dfe52068613 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:52.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.698+0000 7fa23f7fe640 1 -- 172.21.15.16:0/4168293781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234037070 con 0x7fa2400d0cc0 2024-09-12T01:33:52.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.698+0000 7fa23f7fe640 1 -- 172.21.15.16:0/4168293781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa234032440 con 0x7fa2400d0cc0 2024-09-12T01:33:52.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.698+0000 7fa23f7fe640 1 -- 172.21.15.16:0/4168293781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234032810 con 0x7fa2400d0cc0 2024-09-12T01:33:52.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.698+0000 7fa247103640 1 -- 172.21.15.16:0/4168293781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2400d0cc0 msgr2=0x7fa2400d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.698+0000 7fa247103640 1 --2- 172.21.15.16:0/4168293781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2400d0cc0 0x7fa2400d1100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa2400d40a0 tx=0x7fa23402a1a0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.698+0000 7fa247103640 1 -- 172.21.15.16:0/4168293781 shutdown_connections 2024-09-12T01:33:52.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.698+0000 7fa247103640 1 --2- 172.21.15.16:0/4168293781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2400d0cc0 0x7fa2400d1100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.698+0000 7fa247103640 1 --2- 172.21.15.16:0/4168293781 >> v2:172.21.15.33:3300/0 conn(0x7fa2400b76c0 0x7fa2400b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.698+0000 7fa247103640 1 -- 172.21.15.16:0/4168293781 >> 172.21.15.16:0/4168293781 conn(0x7fa24006d1c0 msgr2=0x7fa240073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.698+0000 7fa247103640 1 -- 172.21.15.16:0/4168293781 shutdown_connections 2024-09-12T01:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.698+0000 7fa247103640 1 -- 172.21.15.16:0/4168293781 wait complete. 2024-09-12T01:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.700+0000 7fa247103640 1 Processor -- start 2024-09-12T01:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.700+0000 7fa247103640 1 -- start start 2024-09-12T01:33:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.700+0000 7fa247103640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2400b76c0 0x7fa2400c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.700+0000 7fa247103640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa2400d0cc0 0x7fa2401528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.700+0000 7fa247103640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2401619e0 0x7fa2400caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.700+0000 7fa247103640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2400d1640 con 0x7fa2401619e0 2024-09-12T01:33:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.700+0000 7fa247103640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2400d3bb0 con 0x7fa2400b76c0 2024-09-12T01:33:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.700+0000 7fa247103640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa2400d2250 con 0x7fa2400d0cc0 2024-09-12T01:33:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa2456a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2401619e0 0x7fa2400caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa2456a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2401619e0 0x7fa2400caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46080/0 (socket says 172.21.15.16:46080) 2024-09-12T01:33:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa2456a0640 1 -- 172.21.15.16:0/2217252499 learned_addr learned my addr 172.21.15.16:0/2217252499 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa244e9f640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2400b76c0 0x7fa2400c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa23ffff640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa2400d0cc0 0x7fa2401528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa2456a0640 1 -- 172.21.15.16:0/2217252499 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa2400d0cc0 msgr2=0x7fa2401528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa2456a0640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa2400d0cc0 0x7fa2401528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa2456a0640 1 -- 172.21.15.16:0/2217252499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2400b76c0 msgr2=0x7fa2400c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa2456a0640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2400b76c0 0x7fa2400c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa2456a0640 1 -- 172.21.15.16:0/2217252499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa240156220 con 0x7fa2401619e0 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa244e9f640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2400b76c0 0x7fa2400c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa2456a0640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2401619e0 0x7fa2400caa70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa23800cef0 tx=0x7fa23800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa23d7fa640 1 -- 172.21.15.16:0/2217252499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23801a070 con 0x7fa2401619e0 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa23d7fa640 1 -- 172.21.15.16:0/2217252499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa23800d860 con 0x7fa2401619e0 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa247103640 1 -- 172.21.15.16:0/2217252499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa240156500 con 0x7fa2401619e0 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa23d7fa640 1 -- 172.21.15.16:0/2217252499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23800dc50 con 0x7fa2401619e0 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.701+0000 7fa247103640 1 -- 172.21.15.16:0/2217252499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2400c8fc0 con 0x7fa2401619e0 2024-09-12T01:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.702+0000 7fa247103640 1 -- 172.21.15.16:0/2217252499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2400d2850 con 0x7fa2401619e0 2024-09-12T01:33:52.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.703+0000 7fa23d7fa640 1 -- 172.21.15.16:0/2217252499 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa23801d070 con 0x7fa2401619e0 2024-09-12T01:33:52.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.703+0000 7fa23d7fa640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa22c06be30 0x7fa22c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:52.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.703+0000 7fa23d7fa640 1 -- 172.21.15.16:0/2217252499 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa23808ba80 con 0x7fa2401619e0 2024-09-12T01:33:52.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.703+0000 7fa244e9f640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa22c06be30 0x7fa22c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:52.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.704+0000 7fa244e9f640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa22c06be30 0x7fa22c06e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa230000f60 tx=0x7fa230009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:52.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.708+0000 7fa23d7fa640 1 -- 172.21.15.16:0/2217252499 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa2400d2850 con 0x7fa2401619e0 2024-09-12T01:33:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.893+0000 7fa247103640 1 -- 172.21.15.16:0/2217252499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa2400d1100 con 0x7fa2401619e0 2024-09-12T01:33:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.893+0000 7fa23d7fa640 1 -- 172.21.15.16:0/2217252499 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fa23805a1b0 con 0x7fa2401619e0 2024-09-12T01:33:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-12T01:33:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.895+0000 7fa247103640 1 -- 172.21.15.16:0/2217252499 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa22c06be30 msgr2=0x7fa22c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.895+0000 7fa247103640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa22c06be30 0x7fa22c06e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa230000f60 tx=0x7fa230009040 comp rx=0 tx=0).stop 2024-09-12T01:33:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.895+0000 7fa247103640 1 -- 172.21.15.16:0/2217252499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2401619e0 msgr2=0x7fa2400caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.895+0000 7fa247103640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2401619e0 0x7fa2400caa70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa23800cef0 tx=0x7fa23800a710 comp rx=0 tx=0).stop 2024-09-12T01:33:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.895+0000 7fa247103640 1 -- 172.21.15.16:0/2217252499 shutdown_connections 2024-09-12T01:33:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.895+0000 7fa247103640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa22c06be30 0x7fa22c06e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.895+0000 7fa247103640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2401619e0 0x7fa2400caa70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.895+0000 7fa247103640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa2400d0cc0 0x7fa2401528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.895+0000 7fa247103640 1 --2- 172.21.15.16:0/2217252499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2400b76c0 0x7fa2400c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.895+0000 7fa247103640 1 -- 172.21.15.16:0/2217252499 >> 172.21.15.16:0/2217252499 conn(0x7fa24006d1c0 msgr2=0x7fa2400b8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:52.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.895+0000 7fa247103640 1 -- 172.21.15.16:0/2217252499 shutdown_connections 2024-09-12T01:33:52.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:52.895+0000 7fa247103640 1 -- 172.21.15.16:0/2217252499 wait complete. 2024-09-12T01:33:52.906 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:33:52.906 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:52.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T01:33:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.066+0000 7f38a2576640 1 Processor -- start 2024-09-12T01:33:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.066+0000 7f38a2576640 1 -- start start 2024-09-12T01:33:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.067+0000 7f38a2576640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f389c1619e0 0x7f389c0d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.067+0000 7f38a2576640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f389c0d2d90 0x7f389c0d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.067+0000 7f38a2576640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f389c0d1200 0x7f389c0b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.067+0000 7f38a2576640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f389c0d3f20 con 0x7f389c0d1200 2024-09-12T01:33:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.067+0000 7f38a2576640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f389c0d16b0 con 0x7f389c0d2d90 2024-09-12T01:33:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.067+0000 7f38a2576640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f389c161dc0 con 0x7f389c1619e0 2024-09-12T01:33:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.067+0000 7f38a0b13640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f389c0d1200 0x7f389c0b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.067+0000 7f38a0b13640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f389c0d1200 0x7f389c0b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46084/0 (socket says 172.21.15.16:46084) 2024-09-12T01:33:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.067+0000 7f38a0b13640 1 -- 172.21.15.16:0/1629285027 learned_addr learned my addr 172.21.15.16:0/1629285027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.067+0000 7f3893fff640 1 --2- 172.21.15.16:0/1629285027 >> v2:172.21.15.33:3300/0 conn(0x7f389c0d2d90 0x7f389c0d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.068+0000 7f389bfff640 1 --2- 172.21.15.16:0/1629285027 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f389c1619e0 0x7f389c0d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.068+0000 7f38a0b13640 1 -- 172.21.15.16:0/1629285027 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f389c1619e0 msgr2=0x7f389c0d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.068+0000 7f38a0b13640 1 --2- 172.21.15.16:0/1629285027 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f389c1619e0 0x7f389c0d2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.068+0000 7f38a0b13640 1 -- 172.21.15.16:0/1629285027 >> v2:172.21.15.33:3300/0 conn(0x7f389c0d2d90 msgr2=0x7f389c0d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.068+0000 7f38a0b13640 1 --2- 172.21.15.16:0/1629285027 >> v2:172.21.15.33:3300/0 conn(0x7f389c0d2d90 0x7f389c0d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.068+0000 7f38a0b13640 1 -- 172.21.15.16:0/1629285027 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c0d3310 con 0x7f389c0d1200 2024-09-12T01:33:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.068+0000 7f389bfff640 1 --2- 172.21.15.16:0/1629285027 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f389c1619e0 0x7f389c0d2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.068+0000 7f38a0b13640 1 --2- 172.21.15.16:0/1629285027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f389c0d1200 0x7f389c0b5f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f389c0b70a0 tx=0x7f389402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120886f9938904c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.068+0000 7f389b7fe640 1 -- 172.21.15.16:0/1629285027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894036070 con 0x7f389c0d1200 2024-09-12T01:33:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.069+0000 7f389b7fe640 1 -- 172.21.15.16:0/1629285027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3894031600 con 0x7f389c0d1200 2024-09-12T01:33:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.069+0000 7f389b7fe640 1 -- 172.21.15.16:0/1629285027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940319d0 con 0x7f389c0d1200 2024-09-12T01:33:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.069+0000 7f38a2576640 1 -- 172.21.15.16:0/1629285027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f389c0d1200 msgr2=0x7f389c0b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.069+0000 7f38a2576640 1 --2- 172.21.15.16:0/1629285027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f389c0d1200 0x7f389c0b5f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f389c0b70a0 tx=0x7f389402a6a0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.069+0000 7f38a2576640 1 -- 172.21.15.16:0/1629285027 shutdown_connections 2024-09-12T01:33:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.069+0000 7f38a2576640 1 --2- 172.21.15.16:0/1629285027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f389c0d1200 0x7f389c0b5f30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.069+0000 7f38a2576640 1 --2- 172.21.15.16:0/1629285027 >> v2:172.21.15.33:3300/0 conn(0x7f389c0d2d90 0x7f389c0d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.069+0000 7f38a2576640 1 --2- 172.21.15.16:0/1629285027 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f389c1619e0 0x7f389c0d2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.069+0000 7f38a2576640 1 -- 172.21.15.16:0/1629285027 >> 172.21.15.16:0/1629285027 conn(0x7f389c06d1c0 msgr2=0x7f389c073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.069+0000 7f38a2576640 1 -- 172.21.15.16:0/1629285027 shutdown_connections 2024-09-12T01:33:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.069+0000 7f38a2576640 1 -- 172.21.15.16:0/1629285027 wait complete. 2024-09-12T01:33:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.070+0000 7f38a2576640 1 Processor -- start 2024-09-12T01:33:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.070+0000 7f38a2576640 1 -- start start 2024-09-12T01:33:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.071+0000 7f38a2576640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f389c0d1200 0x7f389c14f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.071+0000 7f38a2576640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c0d2d90 0x7f389c0b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.071+0000 7f38a2576640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f389c1619e0 0x7f389c0b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.071+0000 7f38a2576640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f389c0d3bb0 con 0x7f389c0d1200 2024-09-12T01:33:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.071+0000 7f38a2576640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f389c0d38d0 con 0x7f389c0d2d90 2024-09-12T01:33:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.071+0000 7f38a2576640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f389c0d2250 con 0x7f389c1619e0 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.071+0000 7f389bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f389c0d1200 0x7f389c14f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.071+0000 7f389bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f389c0d1200 0x7f389c14f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46088/0 (socket says 172.21.15.16:46088) 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.071+0000 7f389bfff640 1 -- 172.21.15.16:0/3140093255 learned_addr learned my addr 172.21.15.16:0/3140093255 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.071+0000 7f38a0b13640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f389c1619e0 0x7f389c0b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.071+0000 7f3893fff640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c0d2d90 0x7f389c0b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.072+0000 7f389bfff640 1 -- 172.21.15.16:0/3140093255 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f389c1619e0 msgr2=0x7f389c0b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.072+0000 7f389bfff640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f389c1619e0 0x7f389c0b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.072+0000 7f389bfff640 1 -- 172.21.15.16:0/3140093255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c0d2d90 msgr2=0x7f389c0b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.072+0000 7f389bfff640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c0d2d90 0x7f389c0b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.072+0000 7f389bfff640 1 -- 172.21.15.16:0/3140093255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c0b4bd0 con 0x7f389c0d1200 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.072+0000 7f389bfff640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f389c0d1200 0x7f389c14f970 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c7f0 tx=0x7f388c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.072+0000 7f38997fa640 1 -- 172.21.15.16:0/3140093255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c015070 con 0x7f389c0d1200 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.072+0000 7f38997fa640 1 -- 172.21.15.16:0/3140093255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388c007820 con 0x7f389c0d1200 2024-09-12T01:33:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.072+0000 7f38a2576640 1 -- 172.21.15.16:0/3140093255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389c0b4eb0 con 0x7f389c0d1200 2024-09-12T01:33:53.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.072+0000 7f38997fa640 1 -- 172.21.15.16:0/3140093255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c007bd0 con 0x7f389c0d1200 2024-09-12T01:33:53.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.072+0000 7f38a2576640 1 -- 172.21.15.16:0/3140093255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389c1594e0 con 0x7f389c0d1200 2024-09-12T01:33:53.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.073+0000 7f38a2576640 1 -- 172.21.15.16:0/3140093255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389c0d0cc0 con 0x7f389c0d1200 2024-09-12T01:33:53.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.073+0000 7f38997fa640 1 -- 172.21.15.16:0/3140093255 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f388c018070 con 0x7f389c0d1200 2024-09-12T01:33:53.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.074+0000 7f38997fa640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f387c06be30 0x7f387c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.074+0000 7f38997fa640 1 -- 172.21.15.16:0/3140093255 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f388c08afc0 con 0x7f389c0d1200 2024-09-12T01:33:53.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.074+0000 7f3893fff640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f387c06be30 0x7f387c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.074+0000 7f3893fff640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f387c06be30 0x7f387c06e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f389c15c300 tx=0x7f3884008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:53.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.078+0000 7f38997fa640 1 -- 172.21.15.16:0/3140093255 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f388c059070 con 0x7f389c0d1200 2024-09-12T01:33:53.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.267+0000 7f38a2576640 1 -- 172.21.15.16:0/3140093255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f389c0ba430 con 0x7f389c0d1200 2024-09-12T01:33:53.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.267+0000 7f38997fa640 1 -- 172.21.15.16:0/3140093255 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f388c05d370 con 0x7f389c0d1200 2024-09-12T01:33:53.267 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-12T01:33:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.269+0000 7f38a2576640 1 -- 172.21.15.16:0/3140093255 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f387c06be30 msgr2=0x7f387c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.269+0000 7f38a2576640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f387c06be30 0x7f387c06e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f389c15c300 tx=0x7f3884008040 comp rx=0 tx=0).stop 2024-09-12T01:33:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.269+0000 7f38a2576640 1 -- 172.21.15.16:0/3140093255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f389c0d1200 msgr2=0x7f389c14f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.269+0000 7f38a2576640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f389c0d1200 0x7f389c14f970 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c7f0 tx=0x7f388c00ccb0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.269+0000 7f38a2576640 1 -- 172.21.15.16:0/3140093255 shutdown_connections 2024-09-12T01:33:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.269+0000 7f38a2576640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f387c06be30 0x7f387c06e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.269+0000 7f38a2576640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f389c1619e0 0x7f389c0b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.269+0000 7f38a2576640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c0d2d90 0x7f389c0b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.269+0000 7f38a2576640 1 --2- 172.21.15.16:0/3140093255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f389c0d1200 0x7f389c14f970 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.269+0000 7f38a2576640 1 -- 172.21.15.16:0/3140093255 >> 172.21.15.16:0/3140093255 conn(0x7f389c06d1c0 msgr2=0x7f389c160a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.270+0000 7f38a2576640 1 -- 172.21.15.16:0/3140093255 shutdown_connections 2024-09-12T01:33:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.270+0000 7f38a2576640 1 -- 172.21.15.16:0/3140093255 wait complete. 2024-09-12T01:33:53.280 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-12T01:33:53.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:53.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-12T01:33:53.307 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:33:53.307 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:33:53.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:53.307 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-12T01:33:53.364 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:33:53.364 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:53.384 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:33:53.384 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:53.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T01:33:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.548+0000 7f95972e6640 1 Processor -- start 2024-09-12T01:33:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.548+0000 7f95972e6640 1 -- start start 2024-09-12T01:33:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.549+0000 7f95972e6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95901619e0 0x7f95900d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.549+0000 7f95972e6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f95900d13d0 0x7f95900b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.549+0000 7f95972e6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9590161dc0 con 0x7f95900d2d90 2024-09-12T01:33:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.549+0000 7f95972e6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f95900d3f20 con 0x7f95901619e0 2024-09-12T01:33:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.549+0000 7f95972e6640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f95900d3310 con 0x7f95900d13d0 2024-09-12T01:33:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.549+0000 7f9595082640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95901619e0 0x7f95900d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.549+0000 7f9594881640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f95900d2d90 0x7f95900d0cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43934/0 (socket says 172.21.15.16:43934) 2024-09-12T01:33:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.549+0000 7f9594881640 1 -- 172.21.15.16:0/1394170396 learned_addr learned my addr 172.21.15.16:0/1394170396 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.549+0000 7f9595082640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95901619e0 0x7f95900d2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46096/0 (socket says 172.21.15.16:46096) 2024-09-12T01:33:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.549+0000 7f9595883640 1 --2- 172.21.15.16:0/1394170396 >> v2:172.21.15.33:3300/0 conn(0x7f95900d13d0 0x7f95900b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.550+0000 7f9595082640 1 -- 172.21.15.16:0/1394170396 >> v2:172.21.15.33:3300/0 conn(0x7f95900d13d0 msgr2=0x7f95900b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.550+0000 7f9595082640 1 --2- 172.21.15.16:0/1394170396 >> v2:172.21.15.33:3300/0 conn(0x7f95900d13d0 0x7f95900b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.550+0000 7f9595082640 1 -- 172.21.15.16:0/1394170396 >> v1:172.21.15.16:6789/0 conn(0x7f95900d2d90 legacy=0x7f95900d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.550+0000 7f9595082640 1 -- 172.21.15.16:0/1394170396 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95900d35f0 con 0x7f95901619e0 2024-09-12T01:33:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.550+0000 7f9587fff640 1 -- 172.21.15.16:0/1394170396 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4013461020 0 0) 0x7f9590161dc0 con 0x7f95900d2d90 2024-09-12T01:33:53.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.550+0000 7f9595082640 1 --2- 172.21.15.16:0/1394170396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95901619e0 0x7f95900d2850 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f95900d1b20 tx=0x7f958c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c4090eb1506403 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:53.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.550+0000 7f9587fff640 1 -- 172.21.15.16:0/1394170396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c037070 con 0x7f95901619e0 2024-09-12T01:33:53.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.551+0000 7f9587fff640 1 -- 172.21.15.16:0/1394170396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f958c032440 con 0x7f95901619e0 2024-09-12T01:33:53.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.551+0000 7f9587fff640 1 -- 172.21.15.16:0/1394170396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c032810 con 0x7f95901619e0 2024-09-12T01:33:53.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.551+0000 7f95972e6640 1 -- 172.21.15.16:0/1394170396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95901619e0 msgr2=0x7f95900d2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.551+0000 7f95972e6640 1 --2- 172.21.15.16:0/1394170396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95901619e0 0x7f95900d2850 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f95900d1b20 tx=0x7f958c02a1a0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.551+0000 7f95972e6640 1 -- 172.21.15.16:0/1394170396 shutdown_connections 2024-09-12T01:33:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.551+0000 7f95972e6640 1 --2- 172.21.15.16:0/1394170396 >> v2:172.21.15.33:3300/0 conn(0x7f95900d13d0 0x7f95900b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.551+0000 7f95972e6640 1 --2- 172.21.15.16:0/1394170396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95901619e0 0x7f95900d2850 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.551+0000 7f95972e6640 1 -- 172.21.15.16:0/1394170396 >> 172.21.15.16:0/1394170396 conn(0x7f959006d1c0 msgr2=0x7f9590073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.551+0000 7f95972e6640 1 -- 172.21.15.16:0/1394170396 shutdown_connections 2024-09-12T01:33:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.551+0000 7f95972e6640 1 -- 172.21.15.16:0/1394170396 wait complete. 2024-09-12T01:33:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.552+0000 7f95972e6640 1 Processor -- start 2024-09-12T01:33:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.552+0000 7f95972e6640 1 -- start start 2024-09-12T01:33:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.553+0000 7f95972e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95900d13d0 0x7f95900b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.553+0000 7f95972e6640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f95900d2d90 0x7f9590152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.553+0000 7f95972e6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95901619e0 0x7f95900ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.553+0000 7f95972e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95900d2250 con 0x7f95900d13d0 2024-09-12T01:33:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.553+0000 7f95972e6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f95900d3bb0 con 0x7f95901619e0 2024-09-12T01:33:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.553+0000 7f95972e6640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f95900b7c00 con 0x7f95900d2d90 2024-09-12T01:33:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.553+0000 7f9595082640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95900d13d0 0x7f95900b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.553+0000 7f9595082640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95900d13d0 0x7f95900b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46100/0 (socket says 172.21.15.16:46100) 2024-09-12T01:33:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.553+0000 7f9595082640 1 -- 172.21.15.16:0/1997755147 learned_addr learned my addr 172.21.15.16:0/1997755147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.553+0000 7f9595883640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95901619e0 0x7f95900ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.553+0000 7f9594881640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f95900d2d90 0x7f9590152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.554+0000 7f9595082640 1 -- 172.21.15.16:0/1997755147 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f95900d2d90 msgr2=0x7f9590152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.554+0000 7f9595082640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f95900d2d90 0x7f9590152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.554+0000 7f9595082640 1 -- 172.21.15.16:0/1997755147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95901619e0 msgr2=0x7f95900ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.554+0000 7f9595082640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95901619e0 0x7f95900ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.554+0000 7f9595082640 1 -- 172.21.15.16:0/1997755147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95900cae30 con 0x7f95900d13d0 2024-09-12T01:33:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.554+0000 7f9595883640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95901619e0 0x7f95900ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.554+0000 7f9595082640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95900d13d0 0x7f95900b69e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f958c000c00 tx=0x7f958c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.554+0000 7f9585ffb640 1 -- 172.21.15.16:0/1997755147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c037040 con 0x7f95900d13d0 2024-09-12T01:33:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.554+0000 7f9585ffb640 1 -- 172.21.15.16:0/1997755147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f958c042070 con 0x7f95900d13d0 2024-09-12T01:33:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.554+0000 7f95972e6640 1 -- 172.21.15.16:0/1997755147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95900b5000 con 0x7f95900d13d0 2024-09-12T01:33:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.554+0000 7f9585ffb640 1 -- 172.21.15.16:0/1997755147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c030bd0 con 0x7f95900d13d0 2024-09-12T01:33:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.554+0000 7f95972e6640 1 -- 172.21.15.16:0/1997755147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95900b55c0 con 0x7f95900d13d0 2024-09-12T01:33:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.555+0000 7f95972e6640 1 -- 172.21.15.16:0/1997755147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95900b76c0 con 0x7f95900d13d0 2024-09-12T01:33:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.556+0000 7f9585ffb640 1 -- 172.21.15.16:0/1997755147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f958c047460 con 0x7f95900d13d0 2024-09-12T01:33:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.556+0000 7f9585ffb640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f957406be30 0x7f957406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.556+0000 7f9585ffb640 1 -- 172.21.15.16:0/1997755147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f958c0abed0 con 0x7f95900d13d0 2024-09-12T01:33:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.556+0000 7f9594881640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f957406be30 0x7f957406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.557+0000 7f9594881640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f957406be30 0x7f957406e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f95900c9720 tx=0x7f9580009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.561+0000 7f9585ffb640 1 -- 172.21.15.16:0/1997755147 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f958c032e20 con 0x7f95900d13d0 2024-09-12T01:33:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.746+0000 7f95972e6640 1 -- 172.21.15.16:0/1997755147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f95900cfdb0 con 0x7f95900d13d0 2024-09-12T01:33:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.747+0000 7f9585ffb640 1 -- 172.21.15.16:0/1997755147 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f958c07a460 con 0x7f95900d13d0 2024-09-12T01:33:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-12T01:33:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.749+0000 7f95972e6640 1 -- 172.21.15.16:0/1997755147 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f957406be30 msgr2=0x7f957406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.749+0000 7f95972e6640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f957406be30 0x7f957406e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f95900c9720 tx=0x7f9580009040 comp rx=0 tx=0).stop 2024-09-12T01:33:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.749+0000 7f95972e6640 1 -- 172.21.15.16:0/1997755147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95900d13d0 msgr2=0x7f95900b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.749+0000 7f95972e6640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95900d13d0 0x7f95900b69e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f958c000c00 tx=0x7f958c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.749+0000 7f95972e6640 1 -- 172.21.15.16:0/1997755147 shutdown_connections 2024-09-12T01:33:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.749+0000 7f95972e6640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f957406be30 0x7f957406e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.749+0000 7f95972e6640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95901619e0 0x7f95900ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.749+0000 7f95972e6640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f95900d2d90 0x7f9590152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.749+0000 7f95972e6640 1 --2- 172.21.15.16:0/1997755147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95900d13d0 0x7f95900b69e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.749+0000 7f95972e6640 1 -- 172.21.15.16:0/1997755147 >> 172.21.15.16:0/1997755147 conn(0x7f959006d1c0 msgr2=0x7f95900b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.749+0000 7f95972e6640 1 -- 172.21.15.16:0/1997755147 shutdown_connections 2024-09-12T01:33:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.749+0000 7f95972e6640 1 -- 172.21.15.16:0/1997755147 wait complete. 2024-09-12T01:33:53.760 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:33:53.760 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:33:53.781 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T01:33:53.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.885+0000 7f0a3f6dd640 1 Processor -- start 2024-09-12T01:33:53.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.885+0000 7f0a3f6dd640 1 -- start start 2024-09-12T01:33:53.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.886+0000 7f0a3f6dd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a380b86d0 0x7f0a38169070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.886+0000 7f0a3f6dd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a38161dc0 con 0x7f0a380d0cc0 2024-09-12T01:33:53.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.886+0000 7f0a3f6dd640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a380d3310 con 0x7f0a381619e0 2024-09-12T01:33:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.886+0000 7f0a3f6dd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0a380d3f20 con 0x7f0a380b86d0 2024-09-12T01:33:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.886+0000 7f0a3cc78640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0a380d0cc0 0x7f0a380d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43936/0 (socket says 172.21.15.16:43936) 2024-09-12T01:33:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.886+0000 7f0a3cc78640 1 -- 172.21.15.16:0/2856621055 learned_addr learned my addr 172.21.15.16:0/2856621055 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.886+0000 7f0a3dc7a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a380b86d0 0x7f0a38169070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.887+0000 7f0a37fff640 1 -- 172.21.15.16:0/2856621055 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4170636289 0 0) 0x7f0a38161dc0 con 0x7f0a380d0cc0 2024-09-12T01:33:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.887+0000 7f0a37fff640 1 -- 172.21.15.16:0/2856621055 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a20003520 con 0x7f0a380d0cc0 2024-09-12T01:33:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.887+0000 7f0a3dc7a640 1 -- 172.21.15.16:0/2856621055 >> v1:172.21.15.33:6789/0 conn(0x7f0a381619e0 legacy=0x7f0a380d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.887+0000 7f0a3dc7a640 1 -- 172.21.15.16:0/2856621055 >> v1:172.21.15.16:6789/0 conn(0x7f0a380d0cc0 legacy=0x7f0a380d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.887+0000 7f0a3dc7a640 1 -- 172.21.15.16:0/2856621055 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a380d35f0 con 0x7f0a380b86d0 2024-09-12T01:33:53.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.887+0000 7f0a3dc7a640 1 --2- 172.21.15.16:0/2856621055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a380b86d0 0x7f0a38169070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a300097a0 tx=0x7f0a3002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bf6cef4a14720d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.888+0000 7f0a37fff640 1 -- 172.21.15.16:0/2856621055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30037070 con 0x7f0a380b86d0 2024-09-12T01:33:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.888+0000 7f0a37fff640 1 -- 172.21.15.16:0/2856621055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a30032440 con 0x7f0a380b86d0 2024-09-12T01:33:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.888+0000 7f0a37fff640 1 -- 172.21.15.16:0/2856621055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30032810 con 0x7f0a380b86d0 2024-09-12T01:33:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.888+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2856621055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a380b86d0 msgr2=0x7f0a38169070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.888+0000 7f0a3f6dd640 1 --2- 172.21.15.16:0/2856621055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a380b86d0 0x7f0a38169070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a300097a0 tx=0x7f0a3002a230 comp rx=0 tx=0).stop 2024-09-12T01:33:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.888+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2856621055 shutdown_connections 2024-09-12T01:33:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.888+0000 7f0a3f6dd640 1 --2- 172.21.15.16:0/2856621055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a380b86d0 0x7f0a38169070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.888+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2856621055 >> 172.21.15.16:0/2856621055 conn(0x7f0a3806d1c0 msgr2=0x7f0a38073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.888+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2856621055 shutdown_connections 2024-09-12T01:33:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.888+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2856621055 wait complete. 2024-09-12T01:33:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.889+0000 7f0a3f6dd640 1 Processor -- start 2024-09-12T01:33:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.889+0000 7f0a3f6dd640 1 -- start start 2024-09-12T01:33:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.890+0000 7f0a3f6dd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0a380b86d0 0x7f0a3815b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.890+0000 7f0a3f6dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a380d0cc0 0x7f0a380b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.890+0000 7f0a3f6dd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a381619e0 0x7f0a3814f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.890+0000 7f0a3f6dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a380d2250 con 0x7f0a380d0cc0 2024-09-12T01:33:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.890+0000 7f0a3f6dd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0a380d3bb0 con 0x7f0a381619e0 2024-09-12T01:33:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.890+0000 7f0a3f6dd640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0a380d2f60 con 0x7f0a380b86d0 2024-09-12T01:33:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.890+0000 7f0a3cc78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a380d0cc0 0x7f0a380b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.890+0000 7f0a3cc78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a380d0cc0 0x7f0a380b4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46112/0 (socket says 172.21.15.16:46112) 2024-09-12T01:33:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.890+0000 7f0a3cc78640 1 -- 172.21.15.16:0/2306217541 learned_addr learned my addr 172.21.15.16:0/2306217541 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.890+0000 7f0a3dc7a640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a381619e0 0x7f0a3814f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.890+0000 7f0a3d479640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0a380b86d0 0x7f0a3815b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.891+0000 7f0a3cc78640 1 -- 172.21.15.16:0/2306217541 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0a380b86d0 msgr2=0x7f0a3815b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.891+0000 7f0a3cc78640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0a380b86d0 0x7f0a3815b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.891+0000 7f0a3cc78640 1 -- 172.21.15.16:0/2306217541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a381619e0 msgr2=0x7f0a3814f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.891+0000 7f0a3cc78640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a381619e0 0x7f0a3814f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.891+0000 7f0a3cc78640 1 -- 172.21.15.16:0/2306217541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a380b50d0 con 0x7f0a380d0cc0 2024-09-12T01:33:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.891+0000 7f0a3d479640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0a380b86d0 0x7f0a3815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.891+0000 7f0a3cc78640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a380d0cc0 0x7f0a380b4b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00c440 tx=0x7f0a2c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.891+0000 7f0a35ffb640 1 -- 172.21.15.16:0/2306217541 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c017070 con 0x7f0a380d0cc0 2024-09-12T01:33:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.891+0000 7f0a35ffb640 1 -- 172.21.15.16:0/2306217541 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a2c005ce0 con 0x7f0a380d0cc0 2024-09-12T01:33:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.891+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2306217541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3815bdf0 con 0x7f0a380d0cc0 2024-09-12T01:33:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.891+0000 7f0a35ffb640 1 -- 172.21.15.16:0/2306217541 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0072b0 con 0x7f0a380d0cc0 2024-09-12T01:33:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.891+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2306217541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a38159680 con 0x7f0a380d0cc0 2024-09-12T01:33:53.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.892+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2306217541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3815c310 con 0x7f0a380d0cc0 2024-09-12T01:33:53.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.893+0000 7f0a35ffb640 1 -- 172.21.15.16:0/2306217541 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a2c01a070 con 0x7f0a380d0cc0 2024-09-12T01:33:53.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.893+0000 7f0a35ffb640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0a2406be60 0x7f0a2406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:53.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.893+0000 7f0a35ffb640 1 -- 172.21.15.16:0/2306217541 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f0a2c08a510 con 0x7f0a380d0cc0 2024-09-12T01:33:53.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.893+0000 7f0a3d479640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0a2406be60 0x7f0a2406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:53.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.894+0000 7f0a3d479640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0a2406be60 0x7f0a2406e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0a28001e00 tx=0x7f0a28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:53.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:53.897+0000 7f0a35ffb640 1 -- 172.21.15.16:0/2306217541 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0a3815c310 con 0x7f0a380d0cc0 2024-09-12T01:33:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.084+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2306217541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0a380d1100 con 0x7f0a380d0cc0 2024-09-12T01:33:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.084+0000 7f0a35ffb640 1 -- 172.21.15.16:0/2306217541 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f0a2c058510 con 0x7f0a380d0cc0 2024-09-12T01:33:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-12T01:33:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.086+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2306217541 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0a2406be60 msgr2=0x7f0a2406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.086+0000 7f0a3f6dd640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0a2406be60 0x7f0a2406e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0a28001e00 tx=0x7f0a28009040 comp rx=0 tx=0).stop 2024-09-12T01:33:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.086+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2306217541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a380d0cc0 msgr2=0x7f0a380b4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.086+0000 7f0a3f6dd640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a380d0cc0 0x7f0a380b4b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00c440 tx=0x7f0a2c00c900 comp rx=0 tx=0).stop 2024-09-12T01:33:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.086+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2306217541 shutdown_connections 2024-09-12T01:33:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.086+0000 7f0a3f6dd640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0a2406be60 0x7f0a2406e320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.086+0000 7f0a3f6dd640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a381619e0 0x7f0a3814f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.086+0000 7f0a3f6dd640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a380d0cc0 0x7f0a380b4b90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.086+0000 7f0a3f6dd640 1 --2- 172.21.15.16:0/2306217541 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0a380b86d0 0x7f0a3815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.086+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2306217541 >> 172.21.15.16:0/2306217541 conn(0x7f0a3806d1c0 msgr2=0x7f0a380b9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.086+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2306217541 shutdown_connections 2024-09-12T01:33:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.086+0000 7f0a3f6dd640 1 -- 172.21.15.16:0/2306217541 wait complete. 2024-09-12T01:33:54.098 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2024-09-12T01:33:54.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:54.099 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-12T01:33:54.130 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:33:54.130 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:33:54.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:33:54.131 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-12T01:33:54.189 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T01:33:54.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:33:54.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.293+0000 7fa414f8d640 1 Processor -- start 2024-09-12T01:33:54.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.293+0000 7fa414f8d640 1 -- start start 2024-09-12T01:33:54.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.294+0000 7fa414f8d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa41015e5a0 0x7fa4100d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.294+0000 7fa414f8d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4100d13d0 0x7fa4100b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.294+0000 7fa414f8d640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4100d32d0 con 0x7fa4100d2d70 2024-09-12T01:33:54.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.294+0000 7fa414f8d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa41015e980 con 0x7fa4100d13d0 2024-09-12T01:33:54.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.294+0000 7fa414f8d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa4100d3ee0 con 0x7fa41015e5a0 2024-09-12T01:33:54.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.294+0000 7fa40ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4100d13d0 0x7fa4100b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.294+0000 7fa40ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4100d13d0 0x7fa4100b76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46116/0 (socket says 172.21.15.16:46116) 2024-09-12T01:33:54.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.294+0000 7fa40ed9d640 1 -- 172.21.15.16:0/1740123034 learned_addr learned my addr 172.21.15.16:0/1740123034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:54.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.294+0000 7fa40e59c640 1 --2- 172.21.15.16:0/1740123034 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa41015e5a0 0x7fa4100d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.294+0000 7fa40ed9d640 1 -- 172.21.15.16:0/1740123034 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa41015e5a0 msgr2=0x7fa4100d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:54.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.294+0000 7fa40ed9d640 1 --2- 172.21.15.16:0/1740123034 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa41015e5a0 0x7fa4100d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.294+0000 7fa40ed9d640 1 -- 172.21.15.16:0/1740123034 >> v1:172.21.15.33:6789/0 conn(0x7fa4100d2d70 legacy=0x7fa4100d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.294+0000 7fa40ed9d640 1 -- 172.21.15.16:0/1740123034 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4100d35b0 con 0x7fa4100d13d0 2024-09-12T01:33:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.295+0000 7fa40ed9d640 1 --2- 172.21.15.16:0/1740123034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4100d13d0 0x7fa4100b76c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa408009e50 tx=0x7fa40802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e522cbdd0a896dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.295+0000 7fa40d59a640 1 -- 172.21.15.16:0/1740123034 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408036070 con 0x7fa4100d13d0 2024-09-12T01:33:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.295+0000 7fa40d59a640 1 -- 172.21.15.16:0/1740123034 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa408031600 con 0x7fa4100d13d0 2024-09-12T01:33:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.295+0000 7fa40d59a640 1 -- 172.21.15.16:0/1740123034 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080319d0 con 0x7fa4100d13d0 2024-09-12T01:33:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.296+0000 7fa414f8d640 1 -- 172.21.15.16:0/1740123034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4100d13d0 msgr2=0x7fa4100b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.296+0000 7fa414f8d640 1 --2- 172.21.15.16:0/1740123034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4100d13d0 0x7fa4100b76c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa408009e50 tx=0x7fa40802a710 comp rx=0 tx=0).stop 2024-09-12T01:33:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.296+0000 7fa414f8d640 1 -- 172.21.15.16:0/1740123034 shutdown_connections 2024-09-12T01:33:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.296+0000 7fa414f8d640 1 --2- 172.21.15.16:0/1740123034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4100d13d0 0x7fa4100b76c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.296+0000 7fa414f8d640 1 --2- 172.21.15.16:0/1740123034 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa41015e5a0 0x7fa4100d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.296+0000 7fa414f8d640 1 -- 172.21.15.16:0/1740123034 >> 172.21.15.16:0/1740123034 conn(0x7fa41006d1c0 msgr2=0x7fa410073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:54.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.296+0000 7fa414f8d640 1 -- 172.21.15.16:0/1740123034 shutdown_connections 2024-09-12T01:33:54.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.296+0000 7fa414f8d640 1 -- 172.21.15.16:0/1740123034 wait complete. 2024-09-12T01:33:54.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.297+0000 7fa414f8d640 1 Processor -- start 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.297+0000 7fa414f8d640 1 -- start start 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa414f8d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4100d13d0 0x7fa4100ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa414f8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4100d2d70 0x7fa4100cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa414f8d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa41015e5a0 0x7fa4101f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa414f8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100d22f0 con 0x7fa4100d2d70 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa414f8d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100d3b70 con 0x7fa4100d13d0 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa414f8d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa4100b7c00 con 0x7fa41015e5a0 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa40dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4100d2d70 0x7fa4100cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa40dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4100d2d70 0x7fa4100cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46132/0 (socket says 172.21.15.16:46132) 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa40dd9b640 1 -- 172.21.15.16:0/544082413 learned_addr learned my addr 172.21.15.16:0/544082413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa40ed9d640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa41015e5a0 0x7fa4101f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa40e59c640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4100d13d0 0x7fa4100ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa40dd9b640 1 -- 172.21.15.16:0/544082413 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa41015e5a0 msgr2=0x7fa4101f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa40dd9b640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa41015e5a0 0x7fa4101f3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa40dd9b640 1 -- 172.21.15.16:0/544082413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4100d13d0 msgr2=0x7fa4100ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa40dd9b640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4100d13d0 0x7fa4100ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.298+0000 7fa40dd9b640 1 -- 172.21.15.16:0/544082413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4100c9bb0 con 0x7fa4100d2d70 2024-09-12T01:33:54.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.299+0000 7fa40dd9b640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4100d2d70 0x7fa4100cac50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa40400c470 tx=0x7fa404001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:54.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.299+0000 7fa3feffd640 1 -- 172.21.15.16:0/544082413 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040052d0 con 0x7fa4100d2d70 2024-09-12T01:33:54.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.299+0000 7fa3feffd640 1 -- 172.21.15.16:0/544082413 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa404005470 con 0x7fa4100d2d70 2024-09-12T01:33:54.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.299+0000 7fa414f8d640 1 -- 172.21.15.16:0/544082413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4100c9e90 con 0x7fa4100d2d70 2024-09-12T01:33:54.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.299+0000 7fa3feffd640 1 -- 172.21.15.16:0/544082413 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404007260 con 0x7fa4100d2d70 2024-09-12T01:33:54.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.299+0000 7fa414f8d640 1 -- 172.21.15.16:0/544082413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4100ca450 con 0x7fa4100d2d70 2024-09-12T01:33:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.300+0000 7fa414f8d640 1 -- 172.21.15.16:0/544082413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa410159a80 con 0x7fa4100d2d70 2024-09-12T01:33:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.300+0000 7fa3feffd640 1 -- 172.21.15.16:0/544082413 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa404007400 con 0x7fa4100d2d70 2024-09-12T01:33:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.301+0000 7fa3feffd640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa3f006be30 0x7fa3f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.301+0000 7fa3feffd640 1 -- 172.21.15.16:0/544082413 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fa40408a640 con 0x7fa4100d2d70 2024-09-12T01:33:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.301+0000 7fa40e59c640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa3f006be30 0x7fa3f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.301+0000 7fa40e59c640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa3f006be30 0x7fa3f006e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa400009aa0 tx=0x7fa400009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.305+0000 7fa3feffd640 1 -- 172.21.15.16:0/544082413 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa404058610 con 0x7fa4100d2d70 2024-09-12T01:33:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.525+0000 7fa414f8d640 1 -- 172.21.15.16:0/544082413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4101597a0 con 0x7fa4100d2d70 2024-09-12T01:33:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.525+0000 7fa3feffd640 1 -- 172.21.15.16:0/544082413 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7fa40405c910 con 0x7fa4100d2d70 2024-09-12T01:33:54.525 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:54.526 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-12T01:33:31:251113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4157213575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4157213575},{"type":"v1","addr":"172.21.15.16:6835","nonce":4157213575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1554674078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1554674078},{"type":"v1","addr":"172.21.15.16:6837","nonce":1554674078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/359404074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":359404074},{"type":"v1","addr":"172.21.15.33:6834","nonce":359404074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:33:11.492995+0000","modified":"2024-09-12T01:33:30.603230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2179301157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2179301157},{"type":"v1","addr":"172.21.15.33:6835","nonce":2179301157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-12T01:33:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-12T01:33:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.528+0000 7fa414f8d640 1 -- 172.21.15.16:0/544082413 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa3f006be30 msgr2=0x7fa3f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.528+0000 7fa414f8d640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa3f006be30 0x7fa3f006e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa400009aa0 tx=0x7fa400009190 comp rx=0 tx=0).stop 2024-09-12T01:33:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.528+0000 7fa414f8d640 1 -- 172.21.15.16:0/544082413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4100d2d70 msgr2=0x7fa4100cac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.528+0000 7fa414f8d640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4100d2d70 0x7fa4100cac50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa40400c470 tx=0x7fa404001e70 comp rx=0 tx=0).stop 2024-09-12T01:33:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.528+0000 7fa414f8d640 1 -- 172.21.15.16:0/544082413 shutdown_connections 2024-09-12T01:33:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.528+0000 7fa414f8d640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa3f006be30 0x7fa3f006e2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.528+0000 7fa414f8d640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa41015e5a0 0x7fa4101f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.528+0000 7fa414f8d640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4100d2d70 0x7fa4100cac50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.528+0000 7fa414f8d640 1 --2- 172.21.15.16:0/544082413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4100d13d0 0x7fa4100ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.528+0000 7fa414f8d640 1 -- 172.21.15.16:0/544082413 >> 172.21.15.16:0/544082413 conn(0x7fa41006d1c0 msgr2=0x7fa4100b8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.528+0000 7fa414f8d640 1 -- 172.21.15.16:0/544082413 shutdown_connections 2024-09-12T01:33:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.528+0000 7fa414f8d640 1 -- 172.21.15.16:0/544082413 wait complete. 2024-09-12T01:33:54.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:33:54.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.641+0000 7f0f46191640 1 Processor -- start 2024-09-12T01:33:54.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.641+0000 7f0f46191640 1 -- start start 2024-09-12T01:33:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.642+0000 7f0f46191640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0f400b88f0 0x7f0f40169150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.642+0000 7f0f46191640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f400d3430 con 0x7f0f4015e510 2024-09-12T01:33:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.642+0000 7f0f46191640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f400d36d0 con 0x7f0f400d0cc0 2024-09-12T01:33:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.642+0000 7f0f46191640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f0f400d39b0 con 0x7f0f400b88f0 2024-09-12T01:33:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.642+0000 7f0f3f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0f4015e510 0x7f0f400ceb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43944/0 (socket says 172.21.15.16:43944) 2024-09-12T01:33:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.642+0000 7f0f3f7fe640 1 -- 172.21.15.16:0/2021243513 learned_addr learned my addr 172.21.15.16:0/2021243513 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3ffff640 1 --2- 172.21.15.16:0/2021243513 >> v2:172.21.15.33:3300/0 conn(0x7f0f400b88f0 0x7f0f40169150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708729476 0 0) 0x7f0f400d3430 con 0x7f0f4015e510 2024-09-12T01:33:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f28003520 con 0x7f0f4015e510 2024-09-12T01:33:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 357071498 0 0) 0x7f0f400d36d0 con 0x7f0f400d0cc0 2024-09-12T01:33:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f400d3430 con 0x7f0f400d0cc0 2024-09-12T01:33:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1722965041 0 0) 0x7f0f28003520 con 0x7f0f4015e510 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 >> v2:172.21.15.33:3300/0 conn(0x7f0f400b88f0 msgr2=0x7f0f40169150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 --2- 172.21.15.16:0/2021243513 >> v2:172.21.15.33:3300/0 conn(0x7f0f400b88f0 0x7f0f40169150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 >> v1:172.21.15.33:6789/0 conn(0x7f0f400d0cc0 legacy=0x7f0f400d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f400d3c90 con 0x7f0f4015e510 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f0f30001670 con 0x7f0f4015e510 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 >> v1:172.21.15.16:6789/0 conn(0x7f0f4015e510 legacy=0x7f0f400ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 --2- 172.21.15.16:0/2021243513 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f28003bb0 0x7f0f28003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 --2- 172.21.15.16:0/2021243513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f28004800 0x7f0f28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 --2- 172.21.15.16:0/2021243513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28005210 0x7f0f2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f400d39b0 con 0x7f0f28005210 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2800c160 con 0x7f0f28004800 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2800c360 con 0x7f0f28003bb0 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3ffff640 1 --2- 172.21.15.16:0/2021243513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28005210 0x7f0f2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3effd640 1 --2- 172.21.15.16:0/2021243513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f28004800 0x7f0f28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3f7fe640 1 --2- 172.21.15.16:0/2021243513 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f28003bb0 0x7f0f28003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3ffff640 1 -- 172.21.15.16:0/2021243513 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f28003bb0 msgr2=0x7f0f28003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.643+0000 7f0f3ffff640 1 --2- 172.21.15.16:0/2021243513 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f28003bb0 0x7f0f28003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.644+0000 7f0f3ffff640 1 -- 172.21.15.16:0/2021243513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f28004800 msgr2=0x7f0f28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.644+0000 7f0f3ffff640 1 --2- 172.21.15.16:0/2021243513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f28004800 0x7f0f28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.644+0000 7f0f3ffff640 1 -- 172.21.15.16:0/2021243513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f28003520 con 0x7f0f28005210 2024-09-12T01:33:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.644+0000 7f0f3f7fe640 1 --2- 172.21.15.16:0/2021243513 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f28003bb0 0x7f0f28003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.644+0000 7f0f3ffff640 1 -- 172.21.15.16:0/2021243513 reap_dead start 2024-09-12T01:33:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.644+0000 7f0f3ffff640 1 --2- 172.21.15.16:0/2021243513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28005210 0x7f0f2800b9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009f90 tx=0x7f0f28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e1e1b2bdcda725 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.644+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38035020 con 0x7f0f28005210 2024-09-12T01:33:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.644+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f38003b90 con 0x7f0f28005210 2024-09-12T01:33:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.644+0000 7f0f3e7fc640 1 -- 172.21.15.16:0/2021243513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38009c10 con 0x7f0f28005210 2024-09-12T01:33:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.645+0000 7f0f46191640 1 -- 172.21.15.16:0/2021243513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28005210 msgr2=0x7f0f2800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.645+0000 7f0f46191640 1 --2- 172.21.15.16:0/2021243513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28005210 0x7f0f2800b9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009f90 tx=0x7f0f28005830 comp rx=0 tx=0).stop 2024-09-12T01:33:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.645+0000 7f0f46191640 1 -- 172.21.15.16:0/2021243513 shutdown_connections 2024-09-12T01:33:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.645+0000 7f0f46191640 1 --2- 172.21.15.16:0/2021243513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28005210 0x7f0f2800b9c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.645+0000 7f0f46191640 1 -- 172.21.15.16:0/2021243513 >> 172.21.15.16:0/2021243513 conn(0x7f0f4006d1a0 msgr2=0x7f0f40073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.645+0000 7f0f46191640 1 -- 172.21.15.16:0/2021243513 shutdown_connections 2024-09-12T01:33:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.645+0000 7f0f46191640 1 -- 172.21.15.16:0/2021243513 wait complete. 2024-09-12T01:33:54.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.646+0000 7f0f46191640 1 Processor -- start 2024-09-12T01:33:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.646+0000 7f0f46191640 1 -- start start 2024-09-12T01:33:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.647+0000 7f0f46191640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f28005210 0x7f0f400b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.647+0000 7f0f46191640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f4015ebf0 0x7f0f4015f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.647+0000 7f0f46191640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f40142b90 0x7f0f40142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.647+0000 7f0f46191640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4015e8f0 con 0x7f0f40142b90 2024-09-12T01:33:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.647+0000 7f0f46191640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f400d22a0 con 0x7f0f28005210 2024-09-12T01:33:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.647+0000 7f0f46191640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4015ea70 con 0x7f0f4015ebf0 2024-09-12T01:33:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.647+0000 7f0f3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f40142b90 0x7f0f40142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.647+0000 7f0f3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f40142b90 0x7f0f40142ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46148/0 (socket says 172.21.15.16:46148) 2024-09-12T01:33:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.647+0000 7f0f3ffff640 1 -- 172.21.15.16:0/776690124 learned_addr learned my addr 172.21.15.16:0/776690124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f3effd640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f4015ebf0 0x7f0f4015f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f3f7fe640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f28005210 0x7f0f400b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f3ffff640 1 -- 172.21.15.16:0/776690124 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f4015ebf0 msgr2=0x7f0f4015f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f3ffff640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f4015ebf0 0x7f0f4015f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f3ffff640 1 -- 172.21.15.16:0/776690124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f28005210 msgr2=0x7f0f400b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f3ffff640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f28005210 0x7f0f400b6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f3ffff640 1 -- 172.21.15.16:0/776690124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f400b5390 con 0x7f0f40142b90 2024-09-12T01:33:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f3effd640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f4015ebf0 0x7f0f4015f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f3f7fe640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f28005210 0x7f0f400b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f3ffff640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f40142b90 0x7f0f40142ff0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e230 tx=0x7f0f38009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f27fff640 1 -- 172.21.15.16:0/776690124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38035070 con 0x7f0f40142b90 2024-09-12T01:33:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f27fff640 1 -- 172.21.15.16:0/776690124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f38041070 con 0x7f0f40142b90 2024-09-12T01:33:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f46191640 1 -- 172.21.15.16:0/776690124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f400b5670 con 0x7f0f40142b90 2024-09-12T01:33:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f27fff640 1 -- 172.21.15.16:0/776690124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38009c10 con 0x7f0f40142b90 2024-09-12T01:33:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.648+0000 7f0f46191640 1 -- 172.21.15.16:0/776690124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f400b5c30 con 0x7f0f40142b90 2024-09-12T01:33:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.649+0000 7f0f46191640 1 -- 172.21.15.16:0/776690124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f400d1120 con 0x7f0f40142b90 2024-09-12T01:33:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.649+0000 7f0f27fff640 1 -- 172.21.15.16:0/776690124 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f38005d10 con 0x7f0f40142b90 2024-09-12T01:33:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.650+0000 7f0f27fff640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0f2c06be30 0x7f0f2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.650+0000 7f0f3f7fe640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0f2c06be30 0x7f0f2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.650+0000 7f0f27fff640 1 -- 172.21.15.16:0/776690124 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f0f380abe80 con 0x7f0f40142b90 2024-09-12T01:33:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.650+0000 7f0f3f7fe640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0f2c06be30 0x7f0f2c06e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0f30001200 tx=0x7f0f30000f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.654+0000 7f0f27fff640 1 -- 172.21.15.16:0/776690124 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0f3807a4a0 con 0x7f0f40142b90 2024-09-12T01:33:54.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.867+0000 7f0f46191640 1 -- 172.21.15.16:0/776690124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f4015aeb0 con 0x7f0f40142b90 2024-09-12T01:33:54.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.867+0000 7f0f27fff640 1 -- 172.21.15.16:0/776690124 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7f0f38079da0 con 0x7f0f40142b90 2024-09-12T01:33:54.867 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:54.868 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-12T01:33:31:251113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4157213575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4157213575},{"type":"v1","addr":"172.21.15.16:6835","nonce":4157213575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1554674078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1554674078},{"type":"v1","addr":"172.21.15.16:6837","nonce":1554674078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/359404074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":359404074},{"type":"v1","addr":"172.21.15.33:6834","nonce":359404074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:33:11.492995+0000","modified":"2024-09-12T01:33:30.603230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2179301157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2179301157},{"type":"v1","addr":"172.21.15.33:6835","nonce":2179301157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-12T01:33:54.868 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-12T01:33:54.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.869+0000 7f0f46191640 1 -- 172.21.15.16:0/776690124 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0f2c06be30 msgr2=0x7f0f2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.869+0000 7f0f46191640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0f2c06be30 0x7f0f2c06e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0f30001200 tx=0x7f0f30000f90 comp rx=0 tx=0).stop 2024-09-12T01:33:54.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.869+0000 7f0f46191640 1 -- 172.21.15.16:0/776690124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f40142b90 msgr2=0x7f0f40142ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.869+0000 7f0f46191640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f40142b90 0x7f0f40142ff0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e230 tx=0x7f0f38009750 comp rx=0 tx=0).stop 2024-09-12T01:33:54.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.869+0000 7f0f46191640 1 -- 172.21.15.16:0/776690124 shutdown_connections 2024-09-12T01:33:54.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.869+0000 7f0f46191640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0f2c06be30 0x7f0f2c06e2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.869+0000 7f0f46191640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f40142b90 0x7f0f40142ff0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.869+0000 7f0f46191640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f4015ebf0 0x7f0f4015f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.869+0000 7f0f46191640 1 --2- 172.21.15.16:0/776690124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f28005210 0x7f0f400b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.869+0000 7f0f46191640 1 -- 172.21.15.16:0/776690124 >> 172.21.15.16:0/776690124 conn(0x7f0f4006d1a0 msgr2=0x7f0f400b8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:54.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.870+0000 7f0f46191640 1 -- 172.21.15.16:0/776690124 shutdown_connections 2024-09-12T01:33:54.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.870+0000 7f0f46191640 1 -- 172.21.15.16:0/776690124 wait complete. 2024-09-12T01:33:54.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:33:54.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.983+0000 7f2750f42640 1 Processor -- start 2024-09-12T01:33:54.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.983+0000 7f2750f42640 1 -- start start 2024-09-12T01:33:54.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.984+0000 7f2750f42640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f274c0cc540 0x7f274c0af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.984+0000 7f2750f42640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f274c0cb930 0x7f274c0cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.984+0000 7f2750f42640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f274c0cb1a0 con 0x7f274c15c2b0 2024-09-12T01:33:54.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.984+0000 7f2750f42640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f274c071c70 con 0x7f274c0cb930 2024-09-12T01:33:54.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.984+0000 7f2750f42640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f274c0cb480 con 0x7f274c0cc540 2024-09-12T01:33:54.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.984+0000 7f2749d9b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f274c0cc540 0x7f274c0af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.984+0000 7f274ad9d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f274c0cb930 0x7f274c0cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.984+0000 7f2749d9b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f274c0cc540 0x7f274c0af060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:49744/0 (socket says 172.21.15.16:49744) 2024-09-12T01:33:54.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.984+0000 7f274a59c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f274c15c2b0 0x7f274c0738c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:49870/0 (socket says 172.21.15.16:49870) 2024-09-12T01:33:54.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.984+0000 7f2749d9b640 1 -- 172.21.15.16:0/1601154805 learned_addr learned my addr 172.21.15.16:0/1601154805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:54.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.985+0000 7f2749d9b640 1 -- 172.21.15.16:0/1601154805 >> v2:172.21.15.33:3300/0 conn(0x7f274c0cb930 msgr2=0x7f274c0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:54.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.985+0000 7f2749d9b640 1 --2- 172.21.15.16:0/1601154805 >> v2:172.21.15.33:3300/0 conn(0x7f274c0cb930 0x7f274c0cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.985+0000 7f2749d9b640 1 -- 172.21.15.16:0/1601154805 >> v1:172.21.15.33:6789/0 conn(0x7f274c15c2b0 legacy=0x7f274c0738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.985+0000 7f2749d9b640 1 -- 172.21.15.16:0/1601154805 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c0aec80 con 0x7f274c0cc540 2024-09-12T01:33:54.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.985+0000 7f274ad9d640 1 --2- 172.21.15.16:0/1601154805 >> v2:172.21.15.33:3300/0 conn(0x7f274c0cb930 0x7f274c0cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:54.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.986+0000 7f2749d9b640 1 --2- 172.21.15.16:0/1601154805 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f274c0cc540 0x7f274c0af060 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2740009870 tx=0x7f274002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f7e10f79d5bf1c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:54.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.986+0000 7f274959a640 1 -- 172.21.15.16:0/1601154805 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740037070 con 0x7f274c0cc540 2024-09-12T01:33:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.986+0000 7f274959a640 1 -- 172.21.15.16:0/1601154805 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2740032440 con 0x7f274c0cc540 2024-09-12T01:33:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.986+0000 7f274959a640 1 -- 172.21.15.16:0/1601154805 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740032810 con 0x7f274c0cc540 2024-09-12T01:33:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.986+0000 7f2750f42640 1 -- 172.21.15.16:0/1601154805 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f274c0cc540 msgr2=0x7f274c0af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.986+0000 7f2750f42640 1 --2- 172.21.15.16:0/1601154805 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f274c0cc540 0x7f274c0af060 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2740009870 tx=0x7f274002a270 comp rx=0 tx=0).stop 2024-09-12T01:33:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.986+0000 7f2750f42640 1 -- 172.21.15.16:0/1601154805 shutdown_connections 2024-09-12T01:33:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.986+0000 7f2750f42640 1 --2- 172.21.15.16:0/1601154805 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f274c0cc540 0x7f274c0af060 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.986+0000 7f2750f42640 1 --2- 172.21.15.16:0/1601154805 >> v2:172.21.15.33:3300/0 conn(0x7f274c0cb930 0x7f274c0cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.986+0000 7f2750f42640 1 -- 172.21.15.16:0/1601154805 >> 172.21.15.16:0/1601154805 conn(0x7f274c0cffa0 msgr2=0x7f274c0d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.986+0000 7f2750f42640 1 -- 172.21.15.16:0/1601154805 shutdown_connections 2024-09-12T01:33:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.986+0000 7f2750f42640 1 -- 172.21.15.16:0/1601154805 wait complete. 2024-09-12T01:33:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.988+0000 7f2750f42640 1 Processor -- start 2024-09-12T01:33:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.988+0000 7f2750f42640 1 -- start start 2024-09-12T01:33:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.988+0000 7f2750f42640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f274c0cb930 0x7f274c0b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.988+0000 7f2750f42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c0cc540 0x7f274c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.988+0000 7f2750f42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c15c2b0 0x7f274c0b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.988+0000 7f2750f42640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0ad160 con 0x7f274c0cc540 2024-09-12T01:33:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.988+0000 7f2750f42640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0acfe0 con 0x7f274c15c2b0 2024-09-12T01:33:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.988+0000 7f2750f42640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f274c0ad2e0 con 0x7f274c0cb930 2024-09-12T01:33:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f2749d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c0cc540 0x7f274c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f2749d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c0cc540 0x7f274c06ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46160/0 (socket says 172.21.15.16:46160) 2024-09-12T01:33:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f2749d9b640 1 -- 172.21.15.16:0/3779780350 learned_addr learned my addr 172.21.15.16:0/3779780350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f274ad9d640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c15c2b0 0x7f274c0b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f274a59c640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f274c0cb930 0x7f274c0b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f2749d9b640 1 -- 172.21.15.16:0/3779780350 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f274c0cb930 msgr2=0x7f274c0b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f2749d9b640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f274c0cb930 0x7f274c0b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f2749d9b640 1 -- 172.21.15.16:0/3779780350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c15c2b0 msgr2=0x7f274c0b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f2749d9b640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c15c2b0 0x7f274c0b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f2749d9b640 1 -- 172.21.15.16:0/3779780350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c0b3eb0 con 0x7f274c0cc540 2024-09-12T01:33:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f2749d9b640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c0cc540 0x7f274c06ec10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2740009870 tx=0x7f27400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f273affd640 1 -- 172.21.15.16:0/3779780350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740041070 con 0x7f274c0cc540 2024-09-12T01:33:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f273affd640 1 -- 172.21.15.16:0/3779780350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f274003c890 con 0x7f274c0cc540 2024-09-12T01:33:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f273affd640 1 -- 172.21.15.16:0/3779780350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740037040 con 0x7f274c0cc540 2024-09-12T01:33:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f2750f42640 1 -- 172.21.15.16:0/3779780350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c0b4fc0 con 0x7f274c0cc540 2024-09-12T01:33:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.989+0000 7f2750f42640 1 -- 172.21.15.16:0/3779780350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c0b3170 con 0x7f274c0cc540 2024-09-12T01:33:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.990+0000 7f2750f42640 1 -- 172.21.15.16:0/3779780350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c155140 con 0x7f274c0cc540 2024-09-12T01:33:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.991+0000 7f273affd640 1 -- 172.21.15.16:0/3779780350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2740044070 con 0x7f274c0cc540 2024-09-12T01:33:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.991+0000 7f273affd640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f272c06be30 0x7f272c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.991+0000 7f273affd640 1 -- 172.21.15.16:0/3779780350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f27400abcb0 con 0x7f274c0cc540 2024-09-12T01:33:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.991+0000 7f274a59c640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f272c06be30 0x7f272c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.992+0000 7f274a59c640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f272c06be30 0x7f272c06e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f273c007fd0 tx=0x7f273c007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:54.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:54.996+0000 7f273affd640 1 -- 172.21.15.16:0/3779780350 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f274c155140 con 0x7f274c0cc540 2024-09-12T01:33:55.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.175+0000 7f2750f42640 1 -- 172.21.15.16:0/3779780350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f274c0ad6b0 con 0x7f274c0cc540 2024-09-12T01:33:55.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.176+0000 7f273affd640 1 -- 172.21.15.16:0/3779780350 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18952 (secure 0 0 0) 0x7f2740079d00 con 0x7f274c0cc540 2024-09-12T01:33:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:55.176 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:33:54.327671+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:33:09.364639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:33:10.478237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:33:14.204624+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:33:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.178+0000 7f2750f42640 1 -- 172.21.15.16:0/3779780350 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f272c06be30 msgr2=0x7f272c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.178+0000 7f2750f42640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f272c06be30 0x7f272c06e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f273c007fd0 tx=0x7f273c007e80 comp rx=0 tx=0).stop 2024-09-12T01:33:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.178+0000 7f2750f42640 1 -- 172.21.15.16:0/3779780350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c0cc540 msgr2=0x7f274c06ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.178+0000 7f2750f42640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c0cc540 0x7f274c06ec10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2740009870 tx=0x7f27400046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.178+0000 7f2750f42640 1 -- 172.21.15.16:0/3779780350 shutdown_connections 2024-09-12T01:33:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.178+0000 7f2750f42640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f272c06be30 0x7f272c06e2f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.178+0000 7f2750f42640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c15c2b0 0x7f274c0b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.178+0000 7f2750f42640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c0cc540 0x7f274c06ec10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.178+0000 7f2750f42640 1 --2- 172.21.15.16:0/3779780350 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f274c0cb930 0x7f274c0b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.178+0000 7f2750f42640 1 -- 172.21.15.16:0/3779780350 >> 172.21.15.16:0/3779780350 conn(0x7f274c0cffa0 msgr2=0x7f274c0b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.178+0000 7f2750f42640 1 -- 172.21.15.16:0/3779780350 shutdown_connections 2024-09-12T01:33:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.178+0000 7f2750f42640 1 -- 172.21.15.16:0/3779780350 wait complete. 2024-09-12T01:33:55.189 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T01:33:55.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:33:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.292+0000 7f86b5aa5640 1 Processor -- start 2024-09-12T01:33:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.292+0000 7f86b5aa5640 1 -- start start 2024-09-12T01:33:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.293+0000 7f86b5aa5640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f86b00d0350 0x7f86b006cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.293+0000 7f86b5aa5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86b015e7f0 con 0x7f86b009aa30 2024-09-12T01:33:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.293+0000 7f86b5aa5640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86b00d1a50 con 0x7f86b00b1930 2024-09-12T01:33:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.293+0000 7f86b5aa5640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f86b00d09b0 con 0x7f86b00d0350 2024-09-12T01:33:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.293+0000 7f86af7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f86b009aa30 0x7f86b009ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43954/0 (socket says 172.21.15.16:43954) 2024-09-12T01:33:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.293+0000 7f86af7fe640 1 -- 172.21.15.16:0/420703875 learned_addr learned my addr 172.21.15.16:0/420703875 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.293+0000 7f86aeffd640 1 --2- 172.21.15.16:0/420703875 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f86b00d0350 0x7f86b006cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 737291402 0 0) 0x7f86b015e7f0 con 0x7f86b009aa30 2024-09-12T01:33:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8698003520 con 0x7f86b009aa30 2024-09-12T01:33:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1553380604 0 0) 0x7f8698003520 con 0x7f86b009aa30 2024-09-12T01:33:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f86b00d0350 msgr2=0x7f86b006cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 --2- 172.21.15.16:0/420703875 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f86b00d0350 0x7f86b006cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 >> v1:172.21.15.33:6789/0 conn(0x7f86b00b1930 legacy=0x7f86b00b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b00d0c90 con 0x7f86b009aa30 2024-09-12T01:33:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f86a0001670 con 0x7f86b009aa30 2024-09-12T01:33:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 >> v1:172.21.15.16:6789/0 conn(0x7f86b009aa30 legacy=0x7f86b009ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 --2- 172.21.15.16:0/420703875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8698003a50 0x7f8698003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 --2- 172.21.15.16:0/420703875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8698004800 0x7f8698004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 --2- 172.21.15.16:0/420703875 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f86b00d0350 0x7f86980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86b00d09b0 con 0x7f8698004800 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f869800ba70 con 0x7f8698003a50 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f869800bc70 con 0x7f86b00d0350 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86aeffd640 1 --2- 172.21.15.16:0/420703875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8698004800 0x7f8698004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86af7fe640 1 --2- 172.21.15.16:0/420703875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8698003a50 0x7f8698003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86affff640 1 --2- 172.21.15.16:0/420703875 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f86b00d0350 0x7f86980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86aeffd640 1 -- 172.21.15.16:0/420703875 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f86b00d0350 msgr2=0x7f86980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86aeffd640 1 --2- 172.21.15.16:0/420703875 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f86b00d0350 0x7f86980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86aeffd640 1 -- 172.21.15.16:0/420703875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8698003a50 msgr2=0x7f8698003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86aeffd640 1 --2- 172.21.15.16:0/420703875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8698003a50 0x7f8698003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.294+0000 7f86aeffd640 1 -- 172.21.15.16:0/420703875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8698003520 con 0x7f8698004800 2024-09-12T01:33:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.295+0000 7f86aeffd640 1 --2- 172.21.15.16:0/420703875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8698004800 0x7f8698004c80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009a90 tx=0x7f8698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e0c682c691c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.295+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4036070 con 0x7f8698004800 2024-09-12T01:33:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.295+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a4031400 con 0x7f8698004800 2024-09-12T01:33:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.295+0000 7f86ae7fc640 1 -- 172.21.15.16:0/420703875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4009690 con 0x7f8698004800 2024-09-12T01:33:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.296+0000 7f86b5aa5640 1 -- 172.21.15.16:0/420703875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8698004800 msgr2=0x7f8698004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.296+0000 7f86b5aa5640 1 --2- 172.21.15.16:0/420703875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8698004800 0x7f8698004c80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009a90 tx=0x7f8698005830 comp rx=0 tx=0).stop 2024-09-12T01:33:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.296+0000 7f86affff640 1 -- 172.21.15.16:0/420703875 reap_dead start 2024-09-12T01:33:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.296+0000 7f86b5aa5640 1 -- 172.21.15.16:0/420703875 shutdown_connections 2024-09-12T01:33:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.296+0000 7f86b5aa5640 1 -- 172.21.15.16:0/420703875 >> 172.21.15.16:0/420703875 conn(0x7f86b00b3cf0 msgr2=0x7f86b00b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.296+0000 7f86b5aa5640 1 -- 172.21.15.16:0/420703875 shutdown_connections 2024-09-12T01:33:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.296+0000 7f86b5aa5640 1 -- 172.21.15.16:0/420703875 wait complete. 2024-09-12T01:33:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.297+0000 7f86b5aa5640 1 Processor -- start 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.297+0000 7f86b5aa5640 1 -- start start 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86b5aa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8698004800 0x7f86b01541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86b5aa5640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f86b00b7d00 0x7f86b0155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86b5aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b01558c0 0x7f86b0070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86b5aa5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86b00d1530 con 0x7f86b01558c0 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86b5aa5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86b00d1250 con 0x7f8698004800 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86b5aa5640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f86b00b24d0 con 0x7f86b00b7d00 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b01558c0 0x7f86b0070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b01558c0 0x7f86b0070ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46178/0 (socket says 172.21.15.16:46178) 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8698004800 0x7f86b01541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86affff640 1 -- 172.21.15.16:0/3139665693 learned_addr learned my addr 172.21.15.16:0/3139665693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86aeffd640 1 --2- 172.21.15.16:0/3139665693 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f86b00b7d00 0x7f86b0155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86affff640 1 -- 172.21.15.16:0/3139665693 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f86b00b7d00 msgr2=0x7f86b0155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86affff640 1 --2- 172.21.15.16:0/3139665693 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f86b00b7d00 0x7f86b0155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86affff640 1 -- 172.21.15.16:0/3139665693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8698004800 msgr2=0x7f86b01541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86affff640 1 --2- 172.21.15.16:0/3139665693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8698004800 0x7f86b01541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.298+0000 7f86affff640 1 -- 172.21.15.16:0/3139665693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b00b69c0 con 0x7f86b01558c0 2024-09-12T01:33:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.299+0000 7f86affff640 1 --2- 172.21.15.16:0/3139665693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b01558c0 0x7f86b0070ec0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f86b00741a0 tx=0x7f86a800ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.299+0000 7f8697fff640 1 -- 172.21.15.16:0/3139665693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8019070 con 0x7f86b01558c0 2024-09-12T01:33:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.299+0000 7f8697fff640 1 -- 172.21.15.16:0/3139665693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a8006e20 con 0x7f86b01558c0 2024-09-12T01:33:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.299+0000 7f86b5aa5640 1 -- 172.21.15.16:0/3139665693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b0154720 con 0x7f86b01558c0 2024-09-12T01:33:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.299+0000 7f8697fff640 1 -- 172.21.15.16:0/3139665693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8003b90 con 0x7f86b01558c0 2024-09-12T01:33:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.299+0000 7f86b5aa5640 1 -- 172.21.15.16:0/3139665693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b00b6f40 con 0x7f86b01558c0 2024-09-12T01:33:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.300+0000 7f86b5aa5640 1 -- 172.21.15.16:0/3139665693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b00b2860 con 0x7f86b01558c0 2024-09-12T01:33:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.300+0000 7f8697fff640 1 -- 172.21.15.16:0/3139665693 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86a801c070 con 0x7f86b01558c0 2024-09-12T01:33:55.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.301+0000 7f8697fff640 1 --2- 172.21.15.16:0/3139665693 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f869c06be30 0x7f869c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.301+0000 7f8697fff640 1 -- 172.21.15.16:0/3139665693 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f86a808a540 con 0x7f86b01558c0 2024-09-12T01:33:55.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.301+0000 7f86af7fe640 1 --2- 172.21.15.16:0/3139665693 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f869c06be30 0x7f869c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.302+0000 7f86af7fe640 1 --2- 172.21.15.16:0/3139665693 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f869c06be30 0x7f869c06e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f86a0000c00 tx=0x7f86a0001c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:55.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.305+0000 7f8697fff640 1 -- 172.21.15.16:0/3139665693 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f86a8058510 con 0x7f86b01558c0 2024-09-12T01:33:55.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.517+0000 7f86b5aa5640 1 -- 172.21.15.16:0/3139665693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86b00b8180 con 0x7f86b01558c0 2024-09-12T01:33:55.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.518+0000 7f8697fff640 1 -- 172.21.15.16:0/3139665693 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7f86a805c810 con 0x7f86b01558c0 2024-09-12T01:33:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:55.518 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-12T01:33:31:251113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4157213575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4157213575},{"type":"v1","addr":"172.21.15.16:6835","nonce":4157213575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1554674078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1554674078},{"type":"v1","addr":"172.21.15.16:6837","nonce":1554674078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/359404074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":359404074},{"type":"v1","addr":"172.21.15.33:6834","nonce":359404074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:33:11.492995+0000","modified":"2024-09-12T01:33:30.603230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2179301157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2179301157},{"type":"v1","addr":"172.21.15.33:6835","nonce":2179301157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-12T01:33:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-12T01:33:55.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.520+0000 7f86b5aa5640 1 -- 172.21.15.16:0/3139665693 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f869c06be30 msgr2=0x7f869c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.520+0000 7f86b5aa5640 1 --2- 172.21.15.16:0/3139665693 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f869c06be30 0x7f869c06e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f86a0000c00 tx=0x7f86a0001c00 comp rx=0 tx=0).stop 2024-09-12T01:33:55.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.520+0000 7f86b5aa5640 1 -- 172.21.15.16:0/3139665693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b01558c0 msgr2=0x7f86b0070ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.520+0000 7f86b5aa5640 1 --2- 172.21.15.16:0/3139665693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b01558c0 0x7f86b0070ec0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f86b00741a0 tx=0x7f86a800ff70 comp rx=0 tx=0).stop 2024-09-12T01:33:55.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.520+0000 7f86b5aa5640 1 -- 172.21.15.16:0/3139665693 shutdown_connections 2024-09-12T01:33:55.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.520+0000 7f86b5aa5640 1 --2- 172.21.15.16:0/3139665693 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f869c06be30 0x7f869c06e2f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.520+0000 7f86b5aa5640 1 --2- 172.21.15.16:0/3139665693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b01558c0 0x7f86b0070ec0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.520+0000 7f86b5aa5640 1 --2- 172.21.15.16:0/3139665693 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f86b00b7d00 0x7f86b0155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.520+0000 7f86b5aa5640 1 --2- 172.21.15.16:0/3139665693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8698004800 0x7f86b01541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.520+0000 7f86b5aa5640 1 -- 172.21.15.16:0/3139665693 >> 172.21.15.16:0/3139665693 conn(0x7f86b00b3cf0 msgr2=0x7f86b00cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.520+0000 7f86b5aa5640 1 -- 172.21.15.16:0/3139665693 shutdown_connections 2024-09-12T01:33:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.520+0000 7f86b5aa5640 1 -- 172.21.15.16:0/3139665693 wait complete. 2024-09-12T01:33:55.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:33:55.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.633+0000 7fa949aac640 1 Processor -- start 2024-09-12T01:33:55.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.634+0000 7fa949aac640 1 -- start start 2024-09-12T01:33:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.634+0000 7fa949aac640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa9440d04b0 0x7fa94406e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.634+0000 7fa949aac640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa944070be0 con 0x7fa94415f5b0 2024-09-12T01:33:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.634+0000 7fa949aac640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94415fa90 con 0x7fa944158cb0 2024-09-12T01:33:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.634+0000 7fa949aac640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa9441621c0 con 0x7fa9440d04b0 2024-09-12T01:33:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa943fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa94415f5b0 0x7fa944071540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43962/0 (socket says 172.21.15.16:43962) 2024-09-12T01:33:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa943fff640 1 -- 172.21.15.16:0/1410631049 learned_addr learned my addr 172.21.15.16:0/1410631049 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa942ffd640 1 --2- 172.21.15.16:0/1410631049 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa9440d04b0 0x7fa94406e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3621776599 0 0) 0x7fa944070be0 con 0x7fa94415f5b0 2024-09-12T01:33:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa92c003520 con 0x7fa94415f5b0 2024-09-12T01:33:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4084785849 0 0) 0x7fa92c003520 con 0x7fa94415f5b0 2024-09-12T01:33:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa9440d04b0 msgr2=0x7fa94406e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 --2- 172.21.15.16:0/1410631049 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa9440d04b0 0x7fa94406e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 >> v1:172.21.15.33:6789/0 conn(0x7fa944158cb0 legacy=0x7fa9440cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9441624a0 con 0x7fa94415f5b0 2024-09-12T01:33:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7fa93c0019f0 con 0x7fa94415f5b0 2024-09-12T01:33:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 >> v1:172.21.15.16:6789/0 conn(0x7fa94415f5b0 legacy=0x7fa944071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 --2- 172.21.15.16:0/1410631049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa92c003a50 0x7fa92c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 --2- 172.21.15.16:0/1410631049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c004800 0x7fa92c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 --2- 172.21.15.16:0/1410631049 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa9440d04b0 0x7fa92c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9441621c0 con 0x7fa92c003a50 2024-09-12T01:33:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c00bac0 con 0x7fa92c004800 2024-09-12T01:33:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa92c00bcc0 con 0x7fa9440d04b0 2024-09-12T01:33:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa9437fe640 1 --2- 172.21.15.16:0/1410631049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa92c003a50 0x7fa92c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa942ffd640 1 --2- 172.21.15.16:0/1410631049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c004800 0x7fa92c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.635+0000 7fa943fff640 1 --2- 172.21.15.16:0/1410631049 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa9440d04b0 0x7fa92c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.636+0000 7fa9437fe640 1 -- 172.21.15.16:0/1410631049 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa9440d04b0 msgr2=0x7fa92c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.636+0000 7fa9437fe640 1 --2- 172.21.15.16:0/1410631049 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa9440d04b0 0x7fa92c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.636+0000 7fa9437fe640 1 -- 172.21.15.16:0/1410631049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c004800 msgr2=0x7fa92c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.636+0000 7fa9437fe640 1 --2- 172.21.15.16:0/1410631049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c004800 0x7fa92c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.636+0000 7fa9437fe640 1 -- 172.21.15.16:0/1410631049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c003520 con 0x7fa92c003a50 2024-09-12T01:33:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.636+0000 7fa942ffd640 1 --2- 172.21.15.16:0/1410631049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c004800 0x7fa92c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.636+0000 7fa9437fe640 1 --2- 172.21.15.16:0/1410631049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa92c003a50 0x7fa92c003e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa934001750 tx=0x7fa92c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d1ab3e393ef133 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:55.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.636+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934001370 con 0x7fa92c003a50 2024-09-12T01:33:55.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.636+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9340052d0 con 0x7fa92c003a50 2024-09-12T01:33:55.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.636+0000 7fa9427fc640 1 -- 172.21.15.16:0/1410631049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93402abe0 con 0x7fa92c003a50 2024-09-12T01:33:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.637+0000 7fa949aac640 1 -- 172.21.15.16:0/1410631049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa92c003a50 msgr2=0x7fa92c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.637+0000 7fa949aac640 1 --2- 172.21.15.16:0/1410631049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa92c003a50 0x7fa92c003e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa934001750 tx=0x7fa92c005830 comp rx=0 tx=0).stop 2024-09-12T01:33:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.637+0000 7fa943fff640 1 -- 172.21.15.16:0/1410631049 reap_dead start 2024-09-12T01:33:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.637+0000 7fa949aac640 1 -- 172.21.15.16:0/1410631049 shutdown_connections 2024-09-12T01:33:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.637+0000 7fa949aac640 1 -- 172.21.15.16:0/1410631049 >> 172.21.15.16:0/1410631049 conn(0x7fa94406fdc0 msgr2=0x7fa9440701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.637+0000 7fa949aac640 1 -- 172.21.15.16:0/1410631049 shutdown_connections 2024-09-12T01:33:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.637+0000 7fa949aac640 1 -- 172.21.15.16:0/1410631049 wait complete. 2024-09-12T01:33:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.638+0000 7fa949aac640 1 Processor -- start 2024-09-12T01:33:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.639+0000 7fa949aac640 1 -- start start 2024-09-12T01:33:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.639+0000 7fa949aac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c003a50 0x7fa9441478f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.639+0000 7fa949aac640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa9441460a0 0x7fa944146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.639+0000 7fa949aac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944146a60 0x7fa9440cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.639+0000 7fa949aac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa944162f80 con 0x7fa944146a60 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.639+0000 7fa949aac640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa944162a60 con 0x7fa92c003a50 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.639+0000 7fa949aac640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa94406e780 con 0x7fa9441460a0 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.639+0000 7fa943fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944146a60 0x7fa9440cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.639+0000 7fa943fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944146a60 0x7fa9440cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46194/0 (socket says 172.21.15.16:46194) 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.639+0000 7fa943fff640 1 -- 172.21.15.16:0/3172938894 learned_addr learned my addr 172.21.15.16:0/3172938894 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.639+0000 7fa9437fe640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c003a50 0x7fa9441478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.639+0000 7fa942ffd640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa9441460a0 0x7fa944146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.640+0000 7fa943fff640 1 -- 172.21.15.16:0/3172938894 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa9441460a0 msgr2=0x7fa944146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.640+0000 7fa943fff640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa9441460a0 0x7fa944146520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.640+0000 7fa943fff640 1 -- 172.21.15.16:0/3172938894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c003a50 msgr2=0x7fa9441478f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.640+0000 7fa943fff640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c003a50 0x7fa9441478f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.640+0000 7fa943fff640 1 -- 172.21.15.16:0/3172938894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9440cc8a0 con 0x7fa944146a60 2024-09-12T01:33:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.640+0000 7fa943fff640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944146a60 0x7fa9440cd940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa9440cfa30 tx=0x7fa93c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.640+0000 7fa92bfff640 1 -- 172.21.15.16:0/3172938894 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00eae0 con 0x7fa944146a60 2024-09-12T01:33:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.640+0000 7fa92bfff640 1 -- 172.21.15.16:0/3172938894 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93c018070 con 0x7fa944146a60 2024-09-12T01:33:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.640+0000 7fa949aac640 1 -- 172.21.15.16:0/3172938894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9440ccb80 con 0x7fa944146a60 2024-09-12T01:33:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.640+0000 7fa92bfff640 1 -- 172.21.15.16:0/3172938894 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c013420 con 0x7fa944146a60 2024-09-12T01:33:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.640+0000 7fa949aac640 1 -- 172.21.15.16:0/3172938894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9440cd140 con 0x7fa944146a60 2024-09-12T01:33:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.641+0000 7fa949aac640 1 -- 172.21.15.16:0/3172938894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94406eb60 con 0x7fa944146a60 2024-09-12T01:33:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.642+0000 7fa92bfff640 1 -- 172.21.15.16:0/3172938894 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa93c0135c0 con 0x7fa944146a60 2024-09-12T01:33:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.642+0000 7fa92bfff640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa93006be60 0x7fa93006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.642+0000 7fa92bfff640 1 -- 172.21.15.16:0/3172938894 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fa93c08b320 con 0x7fa944146a60 2024-09-12T01:33:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.642+0000 7fa9437fe640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa93006be60 0x7fa93006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.643+0000 7fa9437fe640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa93006be60 0x7fa93006e320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa934032000 tx=0x7fa93402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.646+0000 7fa92bfff640 1 -- 172.21.15.16:0/3172938894 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa93c059260 con 0x7fa944146a60 2024-09-12T01:33:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.861+0000 7fa949aac640 1 -- 172.21.15.16:0/3172938894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9440b8150 con 0x7fa944146a60 2024-09-12T01:33:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.862+0000 7fa92bfff640 1 -- 172.21.15.16:0/3172938894 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7fa93c05d560 con 0x7fa944146a60 2024-09-12T01:33:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:55.861 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-12T01:33:31:251113+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4157213575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4157213575},{"type":"v1","addr":"172.21.15.16:6835","nonce":4157213575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1554674078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1554674078},{"type":"v1","addr":"172.21.15.16:6837","nonce":1554674078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/359404074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":359404074},{"type":"v1","addr":"172.21.15.33:6834","nonce":359404074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:33:11.492995+0000","modified":"2024-09-12T01:33:30.603230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2179301157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2179301157},{"type":"v1","addr":"172.21.15.33:6835","nonce":2179301157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-12T01:33:55.862 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-12T01:33:55.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.863+0000 7fa949aac640 1 -- 172.21.15.16:0/3172938894 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa93006be60 msgr2=0x7fa93006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.863+0000 7fa949aac640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa93006be60 0x7fa93006e320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa934032000 tx=0x7fa93402e000 comp rx=0 tx=0).stop 2024-09-12T01:33:55.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.863+0000 7fa949aac640 1 -- 172.21.15.16:0/3172938894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944146a60 msgr2=0x7fa9440cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.863+0000 7fa949aac640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944146a60 0x7fa9440cd940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa9440cfa30 tx=0x7fa93c007bc0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.863+0000 7fa949aac640 1 -- 172.21.15.16:0/3172938894 shutdown_connections 2024-09-12T01:33:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.863+0000 7fa949aac640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa93006be60 0x7fa93006e320 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.863+0000 7fa949aac640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944146a60 0x7fa9440cd940 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.863+0000 7fa949aac640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa9441460a0 0x7fa944146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.863+0000 7fa949aac640 1 --2- 172.21.15.16:0/3172938894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c003a50 0x7fa9441478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.863+0000 7fa949aac640 1 -- 172.21.15.16:0/3172938894 >> 172.21.15.16:0/3172938894 conn(0x7fa94406fdc0 msgr2=0x7fa94406d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.864+0000 7fa949aac640 1 -- 172.21.15.16:0/3172938894 shutdown_connections 2024-09-12T01:33:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.864+0000 7fa949aac640 1 -- 172.21.15.16:0/3172938894 wait complete. 2024-09-12T01:33:55.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:33:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.977+0000 7f3311f33640 1 Processor -- start 2024-09-12T01:33:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.977+0000 7f3311f33640 1 -- start start 2024-09-12T01:33:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.978+0000 7f3311f33640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f330c15e510 0x7f330c0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.978+0000 7f3311f33640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f330c0cf040 0x7f330c0d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.978+0000 7f3311f33640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f330c0d16b0 con 0x7f330c0d1200 2024-09-12T01:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.978+0000 7f3311f33640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f330c0d3430 con 0x7f330c0cf040 2024-09-12T01:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.978+0000 7f3311f33640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f330c0d36d0 con 0x7f330c15e510 2024-09-12T01:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.978+0000 7f330bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f330c0d1200 0x7f330c0b5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43968/0 (socket says 172.21.15.16:43968) 2024-09-12T01:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.978+0000 7f330bfff640 1 -- 172.21.15.16:0/3449006852 learned_addr learned my addr 172.21.15.16:0/3449006852 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.978+0000 7f330affd640 1 --2- 172.21.15.16:0/3449006852 >> v2:172.21.15.33:3300/0 conn(0x7f330c0cf040 0x7f330c0d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.978+0000 7f330b7fe640 1 --2- 172.21.15.16:0/3449006852 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f330c15e510 0x7f330c0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.978+0000 7f330a7fc640 1 -- 172.21.15.16:0/3449006852 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 637550785 0 0) 0x7f3304001490 con 0x7f330c0d1200 2024-09-12T01:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.978+0000 7f330a7fc640 1 -- 172.21.15.16:0/3449006852 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32f4003520 con 0x7f330c0d1200 2024-09-12T01:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.979+0000 7f330b7fe640 1 -- 172.21.15.16:0/3449006852 >> v2:172.21.15.33:3300/0 conn(0x7f330c0cf040 msgr2=0x7f330c0d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.979+0000 7f330b7fe640 1 --2- 172.21.15.16:0/3449006852 >> v2:172.21.15.33:3300/0 conn(0x7f330c0cf040 0x7f330c0d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.979+0000 7f330b7fe640 1 -- 172.21.15.16:0/3449006852 >> v1:172.21.15.16:6789/0 conn(0x7f330c0d1200 legacy=0x7f330c0b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.979+0000 7f330b7fe640 1 -- 172.21.15.16:0/3449006852 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c0d39b0 con 0x7f330c15e510 2024-09-12T01:33:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.979+0000 7f330b7fe640 1 --2- 172.21.15.16:0/3449006852 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f330c15e510 0x7f330c0ceb00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0097a0 tx=0x7f32fc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca22da3ac3c2643d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.980+0000 7f330a7fc640 1 -- 172.21.15.16:0/3449006852 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc037070 con 0x7f330c15e510 2024-09-12T01:33:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.980+0000 7f330a7fc640 1 -- 172.21.15.16:0/3449006852 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32fc02ae50 con 0x7f330c15e510 2024-09-12T01:33:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.980+0000 7f330a7fc640 1 -- 172.21.15.16:0/3449006852 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc032640 con 0x7f330c15e510 2024-09-12T01:33:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.980+0000 7f3311f33640 1 -- 172.21.15.16:0/3449006852 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f330c15e510 msgr2=0x7f330c0ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.980+0000 7f3311f33640 1 --2- 172.21.15.16:0/3449006852 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f330c15e510 0x7f330c0ceb00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0097a0 tx=0x7f32fc02a100 comp rx=0 tx=0).stop 2024-09-12T01:33:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.980+0000 7f3311f33640 1 -- 172.21.15.16:0/3449006852 shutdown_connections 2024-09-12T01:33:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.980+0000 7f3311f33640 1 --2- 172.21.15.16:0/3449006852 >> v2:172.21.15.33:3300/0 conn(0x7f330c0cf040 0x7f330c0d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.980+0000 7f3311f33640 1 --2- 172.21.15.16:0/3449006852 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f330c15e510 0x7f330c0ceb00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.980+0000 7f3311f33640 1 -- 172.21.15.16:0/3449006852 >> 172.21.15.16:0/3449006852 conn(0x7f330c06d1a0 msgr2=0x7f330c073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.980+0000 7f3311f33640 1 -- 172.21.15.16:0/3449006852 shutdown_connections 2024-09-12T01:33:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.980+0000 7f3311f33640 1 -- 172.21.15.16:0/3449006852 wait complete. 2024-09-12T01:33:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.981+0000 7f3311f33640 1 Processor -- start 2024-09-12T01:33:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.982+0000 7f3311f33640 1 -- start start 2024-09-12T01:33:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.982+0000 7f3311f33640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f330c0cf040 0x7f330c1522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.982+0000 7f3311f33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330c0d1200 0x7f330c154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.982+0000 7f3311f33640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330c15e510 0x7f330c1f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.982+0000 7f3311f33640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f330c0d22a0 con 0x7f330c0d1200 2024-09-12T01:33:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.982+0000 7f3311f33640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f330c0d3f70 con 0x7f330c15e510 2024-09-12T01:33:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.982+0000 7f3311f33640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f330c15e8f0 con 0x7f330c0cf040 2024-09-12T01:33:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.982+0000 7f330affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330c0d1200 0x7f330c154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.982+0000 7f330affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330c0d1200 0x7f330c154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46206/0 (socket says 172.21.15.16:46206) 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.982+0000 7f330affd640 1 -- 172.21.15.16:0/2367137001 learned_addr learned my addr 172.21.15.16:0/2367137001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f330b7fe640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f330c0cf040 0x7f330c1522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f330bfff640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330c15e510 0x7f330c1f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f330affd640 1 -- 172.21.15.16:0/2367137001 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f330c0cf040 msgr2=0x7f330c1522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f330affd640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f330c0cf040 0x7f330c1522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f330affd640 1 -- 172.21.15.16:0/2367137001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330c15e510 msgr2=0x7f330c1f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f330affd640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330c15e510 0x7f330c1f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f330affd640 1 -- 172.21.15.16:0/2367137001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c1528d0 con 0x7f330c0d1200 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f330b7fe640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f330c0cf040 0x7f330c1522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f330affd640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330c0d1200 0x7f330c154d70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f330000e840 tx=0x7f330000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f32f3fff640 1 -- 172.21.15.16:0/2367137001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300017070 con 0x7f330c0d1200 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f32f3fff640 1 -- 172.21.15.16:0/2367137001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f330000c940 con 0x7f330c0d1200 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f3311f33640 1 -- 172.21.15.16:0/2367137001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330c152bb0 con 0x7f330c0d1200 2024-09-12T01:33:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f32f3fff640 1 -- 172.21.15.16:0/2367137001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330000cd10 con 0x7f330c0d1200 2024-09-12T01:33:55.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.983+0000 7f3311f33640 1 -- 172.21.15.16:0/2367137001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f330c153170 con 0x7f330c0d1200 2024-09-12T01:33:55.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.984+0000 7f3311f33640 1 -- 172.21.15.16:0/2367137001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330c0d0cc0 con 0x7f330c0d1200 2024-09-12T01:33:55.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.985+0000 7f32f3fff640 1 -- 172.21.15.16:0/2367137001 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f330001a070 con 0x7f330c0d1200 2024-09-12T01:33:55.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.985+0000 7f32f3fff640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f32f806be60 0x7f32f806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:55.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.985+0000 7f330b7fe640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f32f806be60 0x7f32f806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:55.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.985+0000 7f32f3fff640 1 -- 172.21.15.16:0/2367137001 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f330008b3a0 con 0x7f330c0d1200 2024-09-12T01:33:55.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.986+0000 7f330b7fe640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f32f806be60 0x7f32f806e320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f32fc034000 tx=0x7f32fc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:55.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:55.990+0000 7f32f3fff640 1 -- 172.21.15.16:0/2367137001 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f33000599c0 con 0x7f330c0d1200 2024-09-12T01:33:56.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.165+0000 7f3311f33640 1 -- 172.21.15.16:0/2367137001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f330c15ac00 con 0x7f330c0d1200 2024-09-12T01:33:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.166+0000 7f32f3fff640 1 -- 172.21.15.16:0/2367137001 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18952 (secure 0 0 0) 0x7f33000592c0 con 0x7f330c0d1200 2024-09-12T01:33:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:33:56.166 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:33:54.327671+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:33:09.364639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:33:10.478237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:33:14.204624+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:33:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.168+0000 7f3311f33640 1 -- 172.21.15.16:0/2367137001 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f32f806be60 msgr2=0x7f32f806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.168+0000 7f3311f33640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f32f806be60 0x7f32f806e320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f32fc034000 tx=0x7f32fc02e000 comp rx=0 tx=0).stop 2024-09-12T01:33:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.168+0000 7f3311f33640 1 -- 172.21.15.16:0/2367137001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330c0d1200 msgr2=0x7f330c154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.168+0000 7f3311f33640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330c0d1200 0x7f330c154d70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f330000e840 tx=0x7f330000ed00 comp rx=0 tx=0).stop 2024-09-12T01:33:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.169+0000 7f3311f33640 1 -- 172.21.15.16:0/2367137001 shutdown_connections 2024-09-12T01:33:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.169+0000 7f3311f33640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f32f806be60 0x7f32f806e320 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.169+0000 7f3311f33640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330c15e510 0x7f330c1f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.169+0000 7f3311f33640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330c0d1200 0x7f330c154d70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.169+0000 7f3311f33640 1 --2- 172.21.15.16:0/2367137001 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f330c0cf040 0x7f330c1522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.169+0000 7f3311f33640 1 -- 172.21.15.16:0/2367137001 >> 172.21.15.16:0/2367137001 conn(0x7f330c06d1a0 msgr2=0x7f330c0744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.169+0000 7f3311f33640 1 -- 172.21.15.16:0/2367137001 shutdown_connections 2024-09-12T01:33:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.169+0000 7f3311f33640 1 -- 172.21.15.16:0/2367137001 wait complete. 2024-09-12T01:33:56.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T01:33:56.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.284+0000 7f9a4ccd0640 1 Processor -- start 2024-09-12T01:33:56.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.284+0000 7f9a4ccd0640 1 -- start start 2024-09-12T01:33:56.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.285+0000 7f9a4ccd0640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9a4815c3d0 0x7f9a480cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:56.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.285+0000 7f9a4ccd0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a480cfd90 0x7f9a480cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:56.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.285+0000 7f9a4ccd0640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9a480ccd50 0x7f9a48071560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:56.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.285+0000 7f9a4ccd0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9a4815c7b0 con 0x7f9a480cfd90 2024-09-12T01:33:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.285+0000 7f9a4ccd0640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f9a480d05a0 con 0x7f9a480ccd50 2024-09-12T01:33:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.285+0000 7f9a4ccd0640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f9a480d0880 con 0x7f9a4815c3d0 2024-09-12T01:33:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.285+0000 7f9a45d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a480cfd90 0x7f9a480cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.285+0000 7f9a45d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a480cfd90 0x7f9a480cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52560/0 (socket says 172.21.15.16:52560) 2024-09-12T01:33:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.285+0000 7f9a45d9b640 1 -- 172.21.15.16:0/2227698474 learned_addr learned my addr 172.21.15.16:0/2227698474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.286+0000 7f9a46d9d640 1 --2- 172.21.15.16:0/2227698474 >> v2:172.21.15.33:3300/0 conn(0x7f9a480ccd50 0x7f9a48071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.286+0000 7f9a4659c640 1 --2- 172.21.15.16:0/2227698474 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9a4815c3d0 0x7f9a480cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.286+0000 7f9a45d9b640 1 -- 172.21.15.16:0/2227698474 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9a4815c3d0 msgr2=0x7f9a480cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.286+0000 7f9a45d9b640 1 --2- 172.21.15.16:0/2227698474 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9a4815c3d0 0x7f9a480cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.286+0000 7f9a45d9b640 1 -- 172.21.15.16:0/2227698474 >> v2:172.21.15.33:3300/0 conn(0x7f9a480ccd50 msgr2=0x7f9a48071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.286+0000 7f9a45d9b640 1 --2- 172.21.15.16:0/2227698474 >> v2:172.21.15.33:3300/0 conn(0x7f9a480ccd50 0x7f9a48071560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.286+0000 7f9a45d9b640 1 -- 172.21.15.16:0/2227698474 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a480d0b60 con 0x7f9a480cfd90 2024-09-12T01:33:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.286+0000 7f9a45d9b640 1 --2- 172.21.15.16:0/2227698474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a480cfd90 0x7f9a480cc810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9a480d1fc0 tx=0x7f9a3002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13763cf2215dc991 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.287+0000 7f9a4559a640 1 -- 172.21.15.16:0/2227698474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30037070 con 0x7f9a480cfd90 2024-09-12T01:33:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.287+0000 7f9a4559a640 1 -- 172.21.15.16:0/2227698474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a30032440 con 0x7f9a480cfd90 2024-09-12T01:33:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.287+0000 7f9a4559a640 1 -- 172.21.15.16:0/2227698474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30032810 con 0x7f9a480cfd90 2024-09-12T01:33:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.287+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2227698474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a480cfd90 msgr2=0x7f9a480cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.287+0000 7f9a4ccd0640 1 --2- 172.21.15.16:0/2227698474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a480cfd90 0x7f9a480cc810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9a480d1fc0 tx=0x7f9a3002a1e0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.287+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2227698474 shutdown_connections 2024-09-12T01:33:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.287+0000 7f9a4ccd0640 1 --2- 172.21.15.16:0/2227698474 >> v2:172.21.15.33:3300/0 conn(0x7f9a480ccd50 0x7f9a48071560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.287+0000 7f9a4ccd0640 1 --2- 172.21.15.16:0/2227698474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a480cfd90 0x7f9a480cc810 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.287+0000 7f9a4ccd0640 1 --2- 172.21.15.16:0/2227698474 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9a4815c3d0 0x7f9a480cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.287+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2227698474 >> 172.21.15.16:0/2227698474 conn(0x7f9a4806fde0 msgr2=0x7f9a480701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.287+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2227698474 shutdown_connections 2024-09-12T01:33:56.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.287+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2227698474 wait complete. 2024-09-12T01:33:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.289+0000 7f9a4ccd0640 1 Processor -- start 2024-09-12T01:33:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.289+0000 7f9a4ccd0640 1 -- start start 2024-09-12T01:33:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.289+0000 7f9a4ccd0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a480ccd50 0x7f9a480ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.289+0000 7f9a4ccd0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a480cfd90 0x7f9a481596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.289+0000 7f9a4ccd0640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9a4815c3d0 0x7f9a480c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.289+0000 7f9a4ccd0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a480d0350 con 0x7f9a480cfd90 2024-09-12T01:33:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.289+0000 7f9a4ccd0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a480d01d0 con 0x7f9a480ccd50 2024-09-12T01:33:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.289+0000 7f9a4ccd0640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f9a48071aa0 con 0x7f9a4815c3d0 2024-09-12T01:33:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.289+0000 7f9a45d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a480cfd90 0x7f9a481596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.289+0000 7f9a45d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a480cfd90 0x7f9a481596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52566/0 (socket says 172.21.15.16:52566) 2024-09-12T01:33:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.289+0000 7f9a45d9b640 1 -- 172.21.15.16:0/2837993620 learned_addr learned my addr 172.21.15.16:0/2837993620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.290+0000 7f9a46d9d640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9a4815c3d0 0x7f9a480c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.290+0000 7f9a4659c640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a480ccd50 0x7f9a480ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.290+0000 7f9a45d9b640 1 -- 172.21.15.16:0/2837993620 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9a4815c3d0 msgr2=0x7f9a480c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:56.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.290+0000 7f9a45d9b640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9a4815c3d0 0x7f9a480c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.290+0000 7f9a45d9b640 1 -- 172.21.15.16:0/2837993620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a480ccd50 msgr2=0x7f9a480ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:56.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.290+0000 7f9a45d9b640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a480ccd50 0x7f9a480ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.290+0000 7f9a45d9b640 1 -- 172.21.15.16:0/2837993620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a480ca9d0 con 0x7f9a480cfd90 2024-09-12T01:33:56.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.290+0000 7f9a45d9b640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a480cfd90 0x7f9a481596c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9a30000c00 tx=0x7f9a300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:56.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.290+0000 7f9a3effd640 1 -- 172.21.15.16:0/2837993620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30041070 con 0x7f9a480cfd90 2024-09-12T01:33:56.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.290+0000 7f9a3effd640 1 -- 172.21.15.16:0/2837993620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a3003c870 con 0x7f9a480cfd90 2024-09-12T01:33:56.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.290+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2837993620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a480c75c0 con 0x7f9a480cfd90 2024-09-12T01:33:56.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.290+0000 7f9a3effd640 1 -- 172.21.15.16:0/2837993620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30037040 con 0x7f9a480cfd90 2024-09-12T01:33:56.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.290+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2837993620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a480c7e00 con 0x7f9a480cfd90 2024-09-12T01:33:56.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.291+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2837993620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a480cc810 con 0x7f9a480cfd90 2024-09-12T01:33:56.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.292+0000 7f9a3effd640 1 -- 172.21.15.16:0/2837993620 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a30044070 con 0x7f9a480cfd90 2024-09-12T01:33:56.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.292+0000 7f9a3effd640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f9a2806be30 0x7f9a2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:56.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.292+0000 7f9a3effd640 1 -- 172.21.15.16:0/2837993620 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f9a300abb50 con 0x7f9a480cfd90 2024-09-12T01:33:56.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.292+0000 7f9a4659c640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f9a2806be30 0x7f9a2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:56.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.293+0000 7f9a4659c640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f9a2806be30 0x7f9a2806e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9a38003aa0 tx=0x7f9a38009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.297+0000 7f9a3effd640 1 -- 172.21.15.16:0/2837993620 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9a480c75c0 con 0x7f9a480cfd90 2024-09-12T01:33:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.507+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2837993620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9a480b3510 con 0x7f9a480cfd90 2024-09-12T01:33:56.507 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-12T01:33:56.508+0000 7fe0e42bd640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T01:33:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.509+0000 7f9a3effd640 1 -- 172.21.15.16:0/2837993620 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f9a30079b20 con 0x7f9a480cfd90 2024-09-12T01:33:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T01:33:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.511+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2837993620 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f9a2806be30 msgr2=0x7f9a2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.511+0000 7f9a4ccd0640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f9a2806be30 0x7f9a2806e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9a38003aa0 tx=0x7f9a38009830 comp rx=0 tx=0).stop 2024-09-12T01:33:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.511+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2837993620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a480cfd90 msgr2=0x7f9a481596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.511+0000 7f9a4ccd0640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a480cfd90 0x7f9a481596c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9a30000c00 tx=0x7f9a300046c0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.512+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2837993620 shutdown_connections 2024-09-12T01:33:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.512+0000 7f9a4ccd0640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f9a2806be30 0x7f9a2806e2f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.512+0000 7f9a4ccd0640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f9a4815c3d0 0x7f9a480c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.512+0000 7f9a4ccd0640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a480cfd90 0x7f9a481596c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.512+0000 7f9a4ccd0640 1 --2- 172.21.15.16:0/2837993620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a480ccd50 0x7f9a480ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.512+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2837993620 >> 172.21.15.16:0/2837993620 conn(0x7f9a4806fde0 msgr2=0x7f9a480b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.512+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2837993620 shutdown_connections 2024-09-12T01:33:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.512+0000 7f9a4ccd0640 1 -- 172.21.15.16:0/2837993620 wait complete. 2024-09-12T01:33:56.515 INFO:tasks.ceph.mds.d.smithi033.stderr: -3256> 2024-09-12T01:33:17.120+0000 7fb6404c9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T01:33:56.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T01:33:56.548 INFO:tasks.ceph.mds.d.smithi033.stdout:starting mds.d at 2024-09-12T01:33:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.629+0000 7eff2341e640 1 Processor -- start 2024-09-12T01:33:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.630+0000 7eff2341e640 1 -- start start 2024-09-12T01:33:56.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.630+0000 7eff2341e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7eff1c0b5c00 0x7eff1c0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:56.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.630+0000 7eff2341e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff1c06baf0 0x7eff1c06bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.630+0000 7eff2341e640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff1c0ae640 con 0x7eff1c161a20 2024-09-12T01:33:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.630+0000 7eff2341e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7eff1c06c490 con 0x7eff1c06baf0 2024-09-12T01:33:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.630+0000 7eff2341e640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7eff1c0b4c40 con 0x7eff1c0b5c00 2024-09-12T01:33:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.630+0000 7eff219bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff1c06baf0 0x7eff1c06bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.631+0000 7eff219bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff1c06baf0 0x7eff1c06bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52580/0 (socket says 172.21.15.16:52580) 2024-09-12T01:33:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.631+0000 7eff219bb640 1 -- 172.21.15.16:0/1816513149 learned_addr learned my addr 172.21.15.16:0/1816513149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.631+0000 7eff209b9640 1 --2- 172.21.15.16:0/1816513149 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7eff1c0b5c00 0x7eff1c0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.631+0000 7eff219bb640 1 -- 172.21.15.16:0/1816513149 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7eff1c0b5c00 msgr2=0x7eff1c0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.631+0000 7eff219bb640 1 --2- 172.21.15.16:0/1816513149 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7eff1c0b5c00 0x7eff1c0706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.631+0000 7eff219bb640 1 -- 172.21.15.16:0/1816513149 >> v1:172.21.15.33:6789/0 conn(0x7eff1c161a20 legacy=0x7eff1c0adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.631+0000 7eff219bb640 1 -- 172.21.15.16:0/1816513149 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c06ee20 con 0x7eff1c06baf0 2024-09-12T01:33:56.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.631+0000 7eff219bb640 1 --2- 172.21.15.16:0/1816513149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c06baf0 0x7eff1c06bf50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009e50 tx=0x7eff0c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e5c97cc089a9d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:56.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.632+0000 7eff1bfff640 1 -- 172.21.15.16:0/1816513149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c036070 con 0x7eff1c06baf0 2024-09-12T01:33:56.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.632+0000 7eff1bfff640 1 -- 172.21.15.16:0/1816513149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff0c031600 con 0x7eff1c06baf0 2024-09-12T01:33:56.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.632+0000 7eff1bfff640 1 -- 172.21.15.16:0/1816513149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c0319d0 con 0x7eff1c06baf0 2024-09-12T01:33:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.632+0000 7eff2341e640 1 -- 172.21.15.16:0/1816513149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c06baf0 msgr2=0x7eff1c06bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.632+0000 7eff2341e640 1 --2- 172.21.15.16:0/1816513149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c06baf0 0x7eff1c06bf50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009e50 tx=0x7eff0c02a680 comp rx=0 tx=0).stop 2024-09-12T01:33:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.632+0000 7eff2341e640 1 -- 172.21.15.16:0/1816513149 shutdown_connections 2024-09-12T01:33:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.632+0000 7eff2341e640 1 --2- 172.21.15.16:0/1816513149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c06baf0 0x7eff1c06bf50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.632+0000 7eff2341e640 1 --2- 172.21.15.16:0/1816513149 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7eff1c0b5c00 0x7eff1c0706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.632+0000 7eff2341e640 1 -- 172.21.15.16:0/1816513149 >> 172.21.15.16:0/1816513149 conn(0x7eff1c0b2c50 msgr2=0x7eff1c0b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.632+0000 7eff2341e640 1 -- 172.21.15.16:0/1816513149 shutdown_connections 2024-09-12T01:33:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.632+0000 7eff2341e640 1 -- 172.21.15.16:0/1816513149 wait complete. 2024-09-12T01:33:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.634+0000 7eff2341e640 1 Processor -- start 2024-09-12T01:33:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.634+0000 7eff2341e640 1 -- start start 2024-09-12T01:33:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.634+0000 7eff2341e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c06baf0 0x7eff1c154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.634+0000 7eff2341e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7eff1c0b5c00 0x7eff1c1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.634+0000 7eff2341e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff1c161a20 0x7eff1c146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.634+0000 7eff2341e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c06f100 con 0x7eff1c06baf0 2024-09-12T01:33:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.634+0000 7eff2341e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c0b4920 con 0x7eff1c161a20 2024-09-12T01:33:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.634+0000 7eff2341e640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7eff1c06f280 con 0x7eff1c0b5c00 2024-09-12T01:33:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff211ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c06baf0 0x7eff1c154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff211ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c06baf0 0x7eff1c154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52590/0 (socket says 172.21.15.16:52590) 2024-09-12T01:33:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff211ba640 1 -- 172.21.15.16:0/4101695614 learned_addr learned my addr 172.21.15.16:0/4101695614 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff209b9640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7eff1c0b5c00 0x7eff1c1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff219bb640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff1c161a20 0x7eff1c146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff211ba640 1 -- 172.21.15.16:0/4101695614 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7eff1c0b5c00 msgr2=0x7eff1c1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff211ba640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7eff1c0b5c00 0x7eff1c1566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff211ba640 1 -- 172.21.15.16:0/4101695614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff1c161a20 msgr2=0x7eff1c146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff211ba640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff1c161a20 0x7eff1c146a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff211ba640 1 -- 172.21.15.16:0/4101695614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c156bf0 con 0x7eff1c06baf0 2024-09-12T01:33:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff211ba640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c06baf0 0x7eff1c154280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7eff1400c3c0 tx=0x7eff1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff19ffb640 1 -- 172.21.15.16:0/4101695614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff140052d0 con 0x7eff1c06baf0 2024-09-12T01:33:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff19ffb640 1 -- 172.21.15.16:0/4101695614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff14005490 con 0x7eff1c06baf0 2024-09-12T01:33:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff2341e640 1 -- 172.21.15.16:0/4101695614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c156e90 con 0x7eff1c06baf0 2024-09-12T01:33:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.635+0000 7eff19ffb640 1 -- 172.21.15.16:0/4101695614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14007260 con 0x7eff1c06baf0 2024-09-12T01:33:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.636+0000 7eff2341e640 1 -- 172.21.15.16:0/4101695614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c157450 con 0x7eff1c06baf0 2024-09-12T01:33:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.637+0000 7eff2341e640 1 -- 172.21.15.16:0/4101695614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff1c06bf50 con 0x7eff1c06baf0 2024-09-12T01:33:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.637+0000 7eff19ffb640 1 -- 172.21.15.16:0/4101695614 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff14007400 con 0x7eff1c06baf0 2024-09-12T01:33:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.637+0000 7eff19ffb640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7eff0806be30 0x7eff0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.637+0000 7eff19ffb640 1 -- 172.21.15.16:0/4101695614 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6850+0+0 (secure 0 0 0) 0x7eff1408a780 con 0x7eff1c06baf0 2024-09-12T01:33:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.637+0000 7eff209b9640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7eff0806be30 0x7eff0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.638+0000 7eff209b9640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7eff0806be30 0x7eff0806e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7eff1c154ad0 tx=0x7eff10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:56.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.642+0000 7eff19ffb640 1 -- 172.21.15.16:0/4101695614 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7eff140585d0 con 0x7eff1c06baf0 2024-09-12T01:33:56.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:56.856+0000 7eff2341e640 1 -- 172.21.15.16:0/4101695614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7eff1c0b5620 con 0x7eff1c06baf0 2024-09-12T01:33:57.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.510+0000 7eff19ffb640 1 -- 172.21.15.16:0/4101695614 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7eff1405c8d0 con 0x7eff1c06baf0 2024-09-12T01:33:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.513+0000 7eff2341e640 1 -- 172.21.15.16:0/4101695614 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7eff0806be30 msgr2=0x7eff0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.513+0000 7eff2341e640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7eff0806be30 0x7eff0806e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7eff1c154ad0 tx=0x7eff10008040 comp rx=0 tx=0).stop 2024-09-12T01:33:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.513+0000 7eff2341e640 1 -- 172.21.15.16:0/4101695614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c06baf0 msgr2=0x7eff1c154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.513+0000 7eff2341e640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c06baf0 0x7eff1c154280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7eff1400c3c0 tx=0x7eff1400c880 comp rx=0 tx=0).stop 2024-09-12T01:33:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.513+0000 7eff2341e640 1 -- 172.21.15.16:0/4101695614 shutdown_connections 2024-09-12T01:33:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.513+0000 7eff2341e640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7eff0806be30 0x7eff0806e2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.513+0000 7eff2341e640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff1c161a20 0x7eff1c146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.513+0000 7eff2341e640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7eff1c0b5c00 0x7eff1c1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.513+0000 7eff2341e640 1 --2- 172.21.15.16:0/4101695614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c06baf0 0x7eff1c154280 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.513+0000 7eff2341e640 1 -- 172.21.15.16:0/4101695614 >> 172.21.15.16:0/4101695614 conn(0x7eff1c0b2c50 msgr2=0x7eff1c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.513+0000 7eff2341e640 1 -- 172.21.15.16:0/4101695614 shutdown_connections 2024-09-12T01:33:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.513+0000 7eff2341e640 1 -- 172.21.15.16:0/4101695614 wait complete. 2024-09-12T01:33:57.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T01:33:57.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.641+0000 7f98cd9af640 1 Processor -- start 2024-09-12T01:33:57.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.641+0000 7f98cd9af640 1 -- start start 2024-09-12T01:33:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.642+0000 7f98cd9af640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f98c815e770 0x7f98c80d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.642+0000 7f98cd9af640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98c8162840 0x7f98c80b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.642+0000 7f98cd9af640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98c80b74c0 con 0x7f98c80d3800 2024-09-12T01:33:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.642+0000 7f98cd9af640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f98c80b7df0 con 0x7f98c8162840 2024-09-12T01:33:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.642+0000 7f98cd9af640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f98c80b77a0 con 0x7f98c815e770 2024-09-12T01:33:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.642+0000 7f98c77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98c8162840 0x7f98c80b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.642+0000 7f98c77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98c8162840 0x7f98c80b9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52592/0 (socket says 172.21.15.16:52592) 2024-09-12T01:33:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.642+0000 7f98bffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f98c80d3800 0x7f98c8162130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48282/0 (socket says 172.21.15.16:48282) 2024-09-12T01:33:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.642+0000 7f98c77fe640 1 -- 172.21.15.16:0/280744369 learned_addr learned my addr 172.21.15.16:0/280744369 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.642+0000 7f98c6ffd640 1 --2- 172.21.15.16:0/280744369 >> v2:172.21.15.33:3300/0 conn(0x7f98c815e770 0x7f98c80d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.642+0000 7f98c77fe640 1 -- 172.21.15.16:0/280744369 >> v2:172.21.15.33:3300/0 conn(0x7f98c815e770 msgr2=0x7f98c80d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.642+0000 7f98c77fe640 1 --2- 172.21.15.16:0/280744369 >> v2:172.21.15.33:3300/0 conn(0x7f98c815e770 0x7f98c80d32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.642+0000 7f98c77fe640 1 -- 172.21.15.16:0/280744369 >> v1:172.21.15.16:6789/0 conn(0x7f98c80d3800 legacy=0x7f98c8162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.643+0000 7f98c67fc640 1 -- 172.21.15.16:0/280744369 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1557556038 0 0) 0x7f98c80b74c0 con 0x7f98c80d3800 2024-09-12T01:33:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.643+0000 7f98c77fe640 1 -- 172.21.15.16:0/280744369 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c80b7a80 con 0x7f98c8162840 2024-09-12T01:33:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.643+0000 7f98c77fe640 1 --2- 172.21.15.16:0/280744369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8162840 0x7f98c80b9a80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f98c8069040 tx=0x7f98c002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbe2894b3bd0cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.643+0000 7f98c67fc640 1 -- 172.21.15.16:0/280744369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0036070 con 0x7f98c8162840 2024-09-12T01:33:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.643+0000 7f98c67fc640 1 -- 172.21.15.16:0/280744369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c0031600 con 0x7f98c8162840 2024-09-12T01:33:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.643+0000 7f98c67fc640 1 -- 172.21.15.16:0/280744369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c00319d0 con 0x7f98c8162840 2024-09-12T01:33:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.644+0000 7f98cd9af640 1 -- 172.21.15.16:0/280744369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8162840 msgr2=0x7f98c80b9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:57.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.644+0000 7f98cd9af640 1 --2- 172.21.15.16:0/280744369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8162840 0x7f98c80b9a80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f98c8069040 tx=0x7f98c002a730 comp rx=0 tx=0).stop 2024-09-12T01:33:57.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.644+0000 7f98cd9af640 1 -- 172.21.15.16:0/280744369 shutdown_connections 2024-09-12T01:33:57.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.644+0000 7f98cd9af640 1 --2- 172.21.15.16:0/280744369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8162840 0x7f98c80b9a80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:57.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.644+0000 7f98cd9af640 1 --2- 172.21.15.16:0/280744369 >> v2:172.21.15.33:3300/0 conn(0x7f98c815e770 0x7f98c80d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:57.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.644+0000 7f98cd9af640 1 -- 172.21.15.16:0/280744369 >> 172.21.15.16:0/280744369 conn(0x7f98c809d660 msgr2=0x7f98c80d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:57.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.644+0000 7f98cd9af640 1 -- 172.21.15.16:0/280744369 shutdown_connections 2024-09-12T01:33:57.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.644+0000 7f98cd9af640 1 -- 172.21.15.16:0/280744369 wait complete. 2024-09-12T01:33:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.645+0000 7f98cd9af640 1 Processor -- start 2024-09-12T01:33:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.645+0000 7f98cd9af640 1 -- start start 2024-09-12T01:33:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.646+0000 7f98cd9af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c80d3800 0x7f98c8153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.646+0000 7f98cd9af640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f98c815e770 0x7f98c80a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.646+0000 7f98cd9af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8162840 0x7f98c8160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.646+0000 7f98cd9af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98c80b9fc0 con 0x7f98c8162840 2024-09-12T01:33:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.646+0000 7f98cd9af640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98c80d53d0 con 0x7f98c80d3800 2024-09-12T01:33:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.646+0000 7f98cd9af640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f98c80ba140 con 0x7f98c815e770 2024-09-12T01:33:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.646+0000 7f98c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8162840 0x7f98c8160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.646+0000 7f98c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8162840 0x7f98c8160d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52608/0 (socket says 172.21.15.16:52608) 2024-09-12T01:33:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.646+0000 7f98c77fe640 1 -- 172.21.15.16:0/1892954561 learned_addr learned my addr 172.21.15.16:0/1892954561 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.646+0000 7f98c6ffd640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c80d3800 0x7f98c8153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.646+0000 7f98bffff640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f98c815e770 0x7f98c80a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.647+0000 7f98c77fe640 1 -- 172.21.15.16:0/1892954561 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f98c815e770 msgr2=0x7f98c80a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.647+0000 7f98c77fe640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f98c815e770 0x7f98c80a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.647+0000 7f98c77fe640 1 -- 172.21.15.16:0/1892954561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c80d3800 msgr2=0x7f98c8153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.647+0000 7f98c77fe640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c80d3800 0x7f98c8153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.647+0000 7f98c77fe640 1 -- 172.21.15.16:0/1892954561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c80a1e40 con 0x7f98c8162840 2024-09-12T01:33:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.647+0000 7f98c6ffd640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c80d3800 0x7f98c8153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.647+0000 7f98c77fe640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8162840 0x7f98c8160d20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009f40 tx=0x7f98c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.647+0000 7f98c4ff9640 1 -- 172.21.15.16:0/1892954561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0036050 con 0x7f98c8162840 2024-09-12T01:33:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.647+0000 7f98c4ff9640 1 -- 172.21.15.16:0/1892954561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c0042070 con 0x7f98c8162840 2024-09-12T01:33:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.647+0000 7f98cd9af640 1 -- 172.21.15.16:0/1892954561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c80c8000 con 0x7f98c8162840 2024-09-12T01:33:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.647+0000 7f98c4ff9640 1 -- 172.21.15.16:0/1892954561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0009510 con 0x7f98c8162840 2024-09-12T01:33:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.647+0000 7f98cd9af640 1 -- 172.21.15.16:0/1892954561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c80c85c0 con 0x7f98c8162840 2024-09-12T01:33:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.648+0000 7f98cd9af640 1 -- 172.21.15.16:0/1892954561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c8069110 con 0x7f98c8162840 2024-09-12T01:33:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.648+0000 7f98c4ff9640 1 -- 172.21.15.16:0/1892954561 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98c0007bf0 con 0x7f98c8162840 2024-09-12T01:33:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.649+0000 7f98c4ff9640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f98a806be30 0x7f98a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.649+0000 7f98c4ff9640 1 -- 172.21.15.16:0/1892954561 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6850+0+0 (secure 0 0 0) 0x7f98c00ac100 con 0x7f98c8162840 2024-09-12T01:33:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.649+0000 7f98c6ffd640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f98a806be30 0x7f98a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.649+0000 7f98c6ffd640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f98a806be30 0x7f98a806e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f98b8006fd0 tx=0x7f98b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.653+0000 7f98c4ff9640 1 -- 172.21.15.16:0/1892954561 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f98c0045360 con 0x7f98c8162840 2024-09-12T01:33:57.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:57.828+0000 7f98cd9af640 1 -- 172.21.15.16:0/1892954561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f98c80d32c0 con 0x7f98c8162840 2024-09-12T01:33:58.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.512+0000 7f98c4ff9640 1 -- 172.21.15.16:0/1892954561 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) ==== 181+0+0 (secure 0 0 0) 0x7f98c007a630 con 0x7f98c8162840 2024-09-12T01:33:58.512 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-12T01:33:58.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.516+0000 7f98cd9af640 1 -- 172.21.15.16:0/1892954561 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f98a806be30 msgr2=0x7f98a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.516+0000 7f98cd9af640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f98a806be30 0x7f98a806e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f98b8006fd0 tx=0x7f98b8008040 comp rx=0 tx=0).stop 2024-09-12T01:33:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.516+0000 7f98cd9af640 1 -- 172.21.15.16:0/1892954561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8162840 msgr2=0x7f98c8160d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.516+0000 7f98cd9af640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8162840 0x7f98c8160d20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009f40 tx=0x7f98c0004a40 comp rx=0 tx=0).stop 2024-09-12T01:33:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.516+0000 7f98cd9af640 1 -- 172.21.15.16:0/1892954561 shutdown_connections 2024-09-12T01:33:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.516+0000 7f98cd9af640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f98a806be30 0x7f98a806e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.516+0000 7f98cd9af640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8162840 0x7f98c8160d20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.516+0000 7f98cd9af640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f98c815e770 0x7f98c80a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.516+0000 7f98cd9af640 1 --2- 172.21.15.16:0/1892954561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c80d3800 0x7f98c8153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.516+0000 7f98cd9af640 1 -- 172.21.15.16:0/1892954561 >> 172.21.15.16:0/1892954561 conn(0x7f98c809d660 msgr2=0x7f98c80d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:58.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.516+0000 7f98cd9af640 1 -- 172.21.15.16:0/1892954561 shutdown_connections 2024-09-12T01:33:58.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.516+0000 7f98cd9af640 1 -- 172.21.15.16:0/1892954561 wait complete. 2024-09-12T01:33:58.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T01:33:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.643+0000 7f889e053640 1 Processor -- start 2024-09-12T01:33:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.643+0000 7f889e053640 1 -- start start 2024-09-12T01:33:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.644+0000 7f889e053640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88980d3050 0x7f88980d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.644+0000 7f889e053640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88980d15e0 con 0x7f88980d11d0 2024-09-12T01:33:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.644+0000 7f889e053640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88980d1fb0 con 0x7f889815e4d0 2024-09-12T01:33:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.644+0000 7f889e053640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f88980d2290 con 0x7f88980d3050 2024-09-12T01:33:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.644+0000 7f888effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88980d3050 0x7f88980d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.644+0000 7f888effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88980d3050 0x7f88980d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52618/0 (socket says 172.21.15.16:52618) 2024-09-12T01:33:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.644+0000 7f888effd640 1 -- 172.21.15.16:0/2797176256 learned_addr learned my addr 172.21.15.16:0/2797176256 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.644+0000 7f8897fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f88980d11d0 0x7f88980b3d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48284/0 (socket says 172.21.15.16:48284) 2024-09-12T01:33:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.645+0000 7f888effd640 1 -- 172.21.15.16:0/2797176256 >> v1:172.21.15.33:6789/0 conn(0x7f889815e4d0 legacy=0x7f88980cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.645+0000 7f888effd640 1 -- 172.21.15.16:0/2797176256 >> v1:172.21.15.16:6789/0 conn(0x7f88980d11d0 legacy=0x7f88980b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.645+0000 7f8896ffd640 1 -- 172.21.15.16:0/2797176256 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2994192153 0 0) 0x7f8890001490 con 0x7f88980d11d0 2024-09-12T01:33:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.645+0000 7f888effd640 1 -- 172.21.15.16:0/2797176256 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88980d2570 con 0x7f88980d3050 2024-09-12T01:33:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.645+0000 7f888effd640 1 --2- 172.21.15.16:0/2797176256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88980d3050 0x7f88980d0c90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8880009870 tx=0x7f888002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c359c1bb661e318 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.646+0000 7f8896ffd640 1 -- 172.21.15.16:0/2797176256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880037070 con 0x7f88980d3050 2024-09-12T01:33:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.646+0000 7f8896ffd640 1 -- 172.21.15.16:0/2797176256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8880032440 con 0x7f88980d3050 2024-09-12T01:33:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.646+0000 7f8896ffd640 1 -- 172.21.15.16:0/2797176256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880032810 con 0x7f88980d3050 2024-09-12T01:33:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.646+0000 7f889e053640 1 -- 172.21.15.16:0/2797176256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88980d3050 msgr2=0x7f88980d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.646+0000 7f889e053640 1 --2- 172.21.15.16:0/2797176256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88980d3050 0x7f88980d0c90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8880009870 tx=0x7f888002a270 comp rx=0 tx=0).stop 2024-09-12T01:33:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.646+0000 7f889e053640 1 -- 172.21.15.16:0/2797176256 shutdown_connections 2024-09-12T01:33:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.646+0000 7f889e053640 1 --2- 172.21.15.16:0/2797176256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88980d3050 0x7f88980d0c90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.646+0000 7f889e053640 1 -- 172.21.15.16:0/2797176256 >> 172.21.15.16:0/2797176256 conn(0x7f889809d650 msgr2=0x7f88980d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.646+0000 7f889e053640 1 -- 172.21.15.16:0/2797176256 shutdown_connections 2024-09-12T01:33:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.646+0000 7f889e053640 1 -- 172.21.15.16:0/2797176256 wait complete. 2024-09-12T01:33:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.647+0000 7f889e053640 1 Processor -- start 2024-09-12T01:33:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.648+0000 7f889e053640 1 -- start start 2024-09-12T01:33:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.648+0000 7f889e053640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f88980d11d0 0x7f88980cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.648+0000 7f889e053640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88980d3050 0x7f889814f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.648+0000 7f889e053640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889815e4d0 0x7f88980cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.648+0000 7f889e053640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88980d4d80 con 0x7f889815e4d0 2024-09-12T01:33:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.648+0000 7f889e053640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f88980d2b30 con 0x7f88980d3050 2024-09-12T01:33:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.648+0000 7f889e053640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f88980d01a0 con 0x7f88980d11d0 2024-09-12T01:33:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.648+0000 7f8897fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889815e4d0 0x7f88980cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f8897fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889815e4d0 0x7f88980cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52632/0 (socket says 172.21.15.16:52632) 2024-09-12T01:33:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f8897fff640 1 -- 172.21.15.16:0/1278018047 learned_addr learned my addr 172.21.15.16:0/1278018047 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f88977fe640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f88980d11d0 0x7f88980cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f888effd640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88980d3050 0x7f889814f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f8897fff640 1 -- 172.21.15.16:0/1278018047 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f88980d11d0 msgr2=0x7f88980cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f8897fff640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f88980d11d0 0x7f88980cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f8897fff640 1 -- 172.21.15.16:0/1278018047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88980d3050 msgr2=0x7f889814f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f8897fff640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88980d3050 0x7f889814f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f8897fff640 1 -- 172.21.15.16:0/1278018047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88980cbfb0 con 0x7f889815e4d0 2024-09-12T01:33:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f8897fff640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889815e4d0 0x7f88980cb4b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f889000c820 tx=0x7f889000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f8894ff9640 1 -- 172.21.15.16:0/1278018047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890018070 con 0x7f889815e4d0 2024-09-12T01:33:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f8894ff9640 1 -- 172.21.15.16:0/1278018047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889000a5e0 con 0x7f889815e4d0 2024-09-12T01:33:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f8894ff9640 1 -- 172.21.15.16:0/1278018047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889000a9d0 con 0x7f889815e4d0 2024-09-12T01:33:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f889e053640 1 -- 172.21.15.16:0/1278018047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88980cc290 con 0x7f889815e4d0 2024-09-12T01:33:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.649+0000 7f889e053640 1 -- 172.21.15.16:0/1278018047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88980cc850 con 0x7f889815e4d0 2024-09-12T01:33:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.650+0000 7f889e053640 1 -- 172.21.15.16:0/1278018047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8898069110 con 0x7f889815e4d0 2024-09-12T01:33:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.651+0000 7f8894ff9640 1 -- 172.21.15.16:0/1278018047 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f889001b070 con 0x7f889815e4d0 2024-09-12T01:33:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.651+0000 7f8894ff9640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f887806be30 0x7f887806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.651+0000 7f8894ff9640 1 -- 172.21.15.16:0/1278018047 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6412+0+0 (secure 0 0 0) 0x7f889008a750 con 0x7f889815e4d0 2024-09-12T01:33:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.651+0000 7f88977fe640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f887806be30 0x7f887806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.652+0000 7f88977fe640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f887806be30 0x7f887806e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8888000f60 tx=0x7f8888009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.656+0000 7f8894ff9640 1 -- 172.21.15.16:0/1278018047 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f88980d15e0 con 0x7f889815e4d0 2024-09-12T01:33:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:58.833+0000 7f889e053640 1 -- 172.21.15.16:0/1278018047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f889815e8b0 con 0x7f889815e4d0 2024-09-12T01:33:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.513+0000 7f8894ff9640 1 -- 172.21.15.16:0/1278018047 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) ==== 169+0+0 (secure 0 0 0) 0x7f8890058760 con 0x7f889815e4d0 2024-09-12T01:33:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-12T01:33:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.517+0000 7f889e053640 1 -- 172.21.15.16:0/1278018047 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f887806be30 msgr2=0x7f887806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.517+0000 7f889e053640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f887806be30 0x7f887806e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8888000f60 tx=0x7f8888009040 comp rx=0 tx=0).stop 2024-09-12T01:33:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.518+0000 7f889e053640 1 -- 172.21.15.16:0/1278018047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889815e4d0 msgr2=0x7f88980cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.518+0000 7f889e053640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889815e4d0 0x7f88980cb4b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f889000c820 tx=0x7f889000cce0 comp rx=0 tx=0).stop 2024-09-12T01:33:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.518+0000 7f889e053640 1 -- 172.21.15.16:0/1278018047 shutdown_connections 2024-09-12T01:33:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.518+0000 7f889e053640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f887806be30 0x7f887806e2f0 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8888000f60 tx=0x7f8888009040 comp rx=0 tx=0).stop 2024-09-12T01:33:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.518+0000 7f889e053640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889815e4d0 0x7f88980cb4b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.518+0000 7f889e053640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88980d3050 0x7f889814f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.518+0000 7f889e053640 1 --2- 172.21.15.16:0/1278018047 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f88980d11d0 0x7f88980cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.518+0000 7f889e053640 1 -- 172.21.15.16:0/1278018047 >> 172.21.15.16:0/1278018047 conn(0x7f889809d650 msgr2=0x7f88980b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.518+0000 7f889e053640 1 -- 172.21.15.16:0/1278018047 shutdown_connections 2024-09-12T01:33:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.518+0000 7f889e053640 1 -- 172.21.15.16:0/1278018047 wait complete. 2024-09-12T01:33:59.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-12T01:33:59.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.648+0000 7f4e813e8640 1 Processor -- start 2024-09-12T01:33:59.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.648+0000 7f4e813e8640 1 -- start start 2024-09-12T01:33:59.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.648+0000 7f4e813e8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e7c162130 0x7f4e7c162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.648+0000 7f4e813e8640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e7c0b9a80 0x7f4e7c0b9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.648+0000 7f4e813e8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e7c0d39d0 con 0x7f4e7c15e770 2024-09-12T01:33:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.648+0000 7f4e813e8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4e7c0b7df0 con 0x7f4e7c162130 2024-09-12T01:33:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.648+0000 7f4e813e8640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f4e7c162ab0 con 0x7f4e7c0b9a80 2024-09-12T01:33:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.649+0000 7f4e7a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e7c162130 0x7f4e7c162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.649+0000 7f4e7a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e7c162130 0x7f4e7c162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52640/0 (socket says 172.21.15.16:52640) 2024-09-12T01:33:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.649+0000 7f4e7a7fc640 1 -- 172.21.15.16:0/1739826410 learned_addr learned my addr 172.21.15.16:0/1739826410 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.649+0000 7f4e7affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4e7c15e770 0x7f4e7c0d32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48290/0 (socket says 172.21.15.16:48290) 2024-09-12T01:33:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.649+0000 7f4e7b7fe640 1 --2- 172.21.15.16:0/1739826410 >> v2:172.21.15.33:3300/0 conn(0x7f4e7c0b9a80 0x7f4e7c0b9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.649+0000 7f4e7a7fc640 1 -- 172.21.15.16:0/1739826410 >> v2:172.21.15.33:3300/0 conn(0x7f4e7c0b9a80 msgr2=0x7f4e7c0b9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.649+0000 7f4e7a7fc640 1 --2- 172.21.15.16:0/1739826410 >> v2:172.21.15.33:3300/0 conn(0x7f4e7c0b9a80 0x7f4e7c0b9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.649+0000 7f4e7a7fc640 1 -- 172.21.15.16:0/1739826410 >> v1:172.21.15.16:6789/0 conn(0x7f4e7c15e770 legacy=0x7f4e7c0d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.649+0000 7f4e7a7fc640 1 -- 172.21.15.16:0/1739826410 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c0b74c0 con 0x7f4e7c162130 2024-09-12T01:33:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.649+0000 7f4e79ffb640 1 -- 172.21.15.16:0/1739826410 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4091390551 0 0) 0x7f4e7c0d39d0 con 0x7f4e7c15e770 2024-09-12T01:33:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.650+0000 7f4e7a7fc640 1 --2- 172.21.15.16:0/1739826410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e7c162130 0x7f4e7c162570 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0cb530 tx=0x7f4e6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3ccbe4cfe9e14f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.650+0000 7f4e79ffb640 1 -- 172.21.15.16:0/1739826410 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64037070 con 0x7f4e7c162130 2024-09-12T01:33:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.650+0000 7f4e79ffb640 1 -- 172.21.15.16:0/1739826410 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e64032440 con 0x7f4e7c162130 2024-09-12T01:33:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.650+0000 7f4e79ffb640 1 -- 172.21.15.16:0/1739826410 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64032810 con 0x7f4e7c162130 2024-09-12T01:33:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.650+0000 7f4e813e8640 1 -- 172.21.15.16:0/1739826410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e7c162130 msgr2=0x7f4e7c162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.650+0000 7f4e813e8640 1 --2- 172.21.15.16:0/1739826410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e7c162130 0x7f4e7c162570 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0cb530 tx=0x7f4e6402a270 comp rx=0 tx=0).stop 2024-09-12T01:33:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.650+0000 7f4e813e8640 1 -- 172.21.15.16:0/1739826410 shutdown_connections 2024-09-12T01:33:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.650+0000 7f4e813e8640 1 --2- 172.21.15.16:0/1739826410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e7c162130 0x7f4e7c162570 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.650+0000 7f4e813e8640 1 --2- 172.21.15.16:0/1739826410 >> v2:172.21.15.33:3300/0 conn(0x7f4e7c0b9a80 0x7f4e7c0b9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.650+0000 7f4e813e8640 1 -- 172.21.15.16:0/1739826410 >> 172.21.15.16:0/1739826410 conn(0x7f4e7c09d660 msgr2=0x7f4e7c0d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.650+0000 7f4e813e8640 1 -- 172.21.15.16:0/1739826410 shutdown_connections 2024-09-12T01:33:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.650+0000 7f4e813e8640 1 -- 172.21.15.16:0/1739826410 wait complete. 2024-09-12T01:33:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.652+0000 7f4e813e8640 1 Processor -- start 2024-09-12T01:33:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.652+0000 7f4e813e8640 1 -- start start 2024-09-12T01:33:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e813e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c0b9a80 0x7f4e7c0c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e813e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e7c15e770 0x7f4e7c160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e813e8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4e7c162130 0x7f4e7c1f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e813e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c0d51c0 con 0x7f4e7c15e770 2024-09-12T01:33:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e813e8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c0b7a80 con 0x7f4e7c0b9a80 2024-09-12T01:33:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e813e8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7c0d5340 con 0x7f4e7c162130 2024-09-12T01:33:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e7a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e7c15e770 0x7f4e7c160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e7a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e7c15e770 0x7f4e7c160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52654/0 (socket says 172.21.15.16:52654) 2024-09-12T01:33:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e7a7fc640 1 -- 172.21.15.16:0/3773983788 learned_addr learned my addr 172.21.15.16:0/3773983788 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:33:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e7affd640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c0b9a80 0x7f4e7c0c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e7b7fe640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4e7c162130 0x7f4e7c1f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e7a7fc640 1 -- 172.21.15.16:0/3773983788 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4e7c162130 msgr2=0x7f4e7c1f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e7a7fc640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4e7c162130 0x7f4e7c1f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e7a7fc640 1 -- 172.21.15.16:0/3773983788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c0b9a80 msgr2=0x7f4e7c0c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e7a7fc640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c0b9a80 0x7f4e7c0c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e7a7fc640 1 -- 172.21.15.16:0/3773983788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c15f720 con 0x7f4e7c15e770 2024-09-12T01:33:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.653+0000 7f4e7b7fe640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4e7c162130 0x7f4e7c1f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.654+0000 7f4e7a7fc640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e7c15e770 0x7f4e7c160ba0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000c00 tx=0x7f4e640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.654+0000 7f4e737fe640 1 -- 172.21.15.16:0/3773983788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64041020 con 0x7f4e7c15e770 2024-09-12T01:33:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.654+0000 7f4e737fe640 1 -- 172.21.15.16:0/3773983788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e640306c0 con 0x7f4e7c15e770 2024-09-12T01:33:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.654+0000 7f4e737fe640 1 -- 172.21.15.16:0/3773983788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64037040 con 0x7f4e7c15e770 2024-09-12T01:33:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.654+0000 7f4e813e8640 1 -- 172.21.15.16:0/3773983788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c15fa00 con 0x7f4e7c15e770 2024-09-12T01:33:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.654+0000 7f4e813e8640 1 -- 172.21.15.16:0/3773983788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c1602a0 con 0x7f4e7c15e770 2024-09-12T01:33:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.655+0000 7f4e813e8640 1 -- 172.21.15.16:0/3773983788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e7c15fdf0 con 0x7f4e7c15e770 2024-09-12T01:33:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.655+0000 7f4e737fe640 1 -- 172.21.15.16:0/3773983788 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e64044020 con 0x7f4e7c15e770 2024-09-12T01:33:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.656+0000 7f4e737fe640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4e5c06be30 0x7f4e5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.656+0000 7f4e737fe640 1 -- 172.21.15.16:0/3773983788 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5853+0+0 (secure 0 0 0) 0x7f4e640ab680 con 0x7f4e7c15e770 2024-09-12T01:33:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.656+0000 7f4e7affd640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4e5c06be30 0x7f4e5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.656+0000 7f4e7affd640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4e5c06be30 0x7f4e5c06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4e74007fd0 tx=0x7f4e74007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.660+0000 7f4e737fe640 1 -- 172.21.15.16:0/3773983788 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4e6404b030 con 0x7f4e7c15e770 2024-09-12T01:33:59.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:33:59.836+0000 7f4e813e8640 1 -- 172.21.15.16:0/3773983788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4e7c069110 con 0x7f4e7c15e770 2024-09-12T01:34:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.525+0000 7f4e737fe640 1 -- 172.21.15.16:0/3773983788 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) ==== 178+0+0 (secure 0 0 0) 0x7f4e64079fb0 con 0x7f4e7c15e770 2024-09-12T01:34:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' removed 2024-09-12T01:34:00.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.530+0000 7f4e813e8640 1 -- 172.21.15.16:0/3773983788 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4e5c06be30 msgr2=0x7f4e5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:00.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.530+0000 7f4e813e8640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4e5c06be30 0x7f4e5c06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4e74007fd0 tx=0x7f4e74007de0 comp rx=0 tx=0).stop 2024-09-12T01:34:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.530+0000 7f4e813e8640 1 -- 172.21.15.16:0/3773983788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e7c15e770 msgr2=0x7f4e7c160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.530+0000 7f4e813e8640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e7c15e770 0x7f4e7c160ba0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000c00 tx=0x7f4e640046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.530+0000 7f4e813e8640 1 -- 172.21.15.16:0/3773983788 shutdown_connections 2024-09-12T01:34:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.530+0000 7f4e813e8640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4e5c06be30 0x7f4e5c06e2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.530+0000 7f4e813e8640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4e7c162130 0x7f4e7c1f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.530+0000 7f4e813e8640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e7c15e770 0x7f4e7c160ba0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.530+0000 7f4e813e8640 1 --2- 172.21.15.16:0/3773983788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c0b9a80 0x7f4e7c0c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.530+0000 7f4e813e8640 1 -- 172.21.15.16:0/3773983788 >> 172.21.15.16:0/3773983788 conn(0x7f4e7c09d660 msgr2=0x7f4e7c06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.530+0000 7f4e813e8640 1 -- 172.21.15.16:0/3773983788 shutdown_connections 2024-09-12T01:34:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.530+0000 7f4e813e8640 1 -- 172.21.15.16:0/3773983788 wait complete. 2024-09-12T01:34:00.546 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T01:34:00.546 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T01:34:00.546 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T01:34:00.546 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:34:00.546 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T01:34:00.546 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T01:34:00.547 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T01:34:00.547 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:34:00.547 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T01:34:00.547 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T01:34:00.547 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T01:34:00.547 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:34:00.547 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T01:34:00.548 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T01:34:00.548 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T01:34:00.548 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:34:00.558 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T01:34:00.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T01:34:00.559 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T01:34:00.559 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T01:34:00.560 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T01:34:00.560 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T01:34:00.561 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T01:34:00.561 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T01:34:00.605 INFO:tasks.ceph.mds.a:Started 2024-09-12T01:34:00.605 DEBUG:teuthology.parallel:result is None 2024-09-12T01:34:00.606 INFO:tasks.ceph.mds.b:Started 2024-09-12T01:34:00.606 DEBUG:teuthology.parallel:result is None 2024-09-12T01:34:00.609 INFO:tasks.ceph.mds.c:Started 2024-09-12T01:34:00.609 DEBUG:teuthology.parallel:result is None 2024-09-12T01:34:00.609 INFO:tasks.ceph.mds.d:Started 2024-09-12T01:34:00.610 DEBUG:teuthology.parallel:result is None 2024-09-12T01:34:00.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T01:34:00.693 INFO:tasks.ceph.mds.b.smithi033.stdout:starting mds.b at 2024-09-12T01:34:00.694 INFO:tasks.ceph.mds.d.smithi033.stdout:starting mds.d at 2024-09-12T01:34:00.695 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-12T01:34:00.695 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-12T01:34:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.730+0000 7f4815801640 1 Processor -- start 2024-09-12T01:34:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.730+0000 7f4815801640 1 -- start start 2024-09-12T01:34:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f4815801640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f481006baf0 0x7f481006bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f4815801640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f481006ff10 0x7f48100703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f4815801640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4810072ee0 con 0x7f481015e260 2024-09-12T01:34:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f4815801640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f481006c490 con 0x7f481006baf0 2024-09-12T01:34:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f4815801640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f48100731c0 con 0x7f481006ff10 2024-09-12T01:34:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f480e7fc640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f481006baf0 0x7f481006bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f480f7fe640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f481006ff10 0x7f48100703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f480f7fe640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f481006ff10 0x7f48100703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:54634/0 (socket says 172.21.15.16:54634) 2024-09-12T01:34:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f480e7fc640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f481006baf0 0x7f481006bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:44972/0 (socket says 172.21.15.16:44972) 2024-09-12T01:34:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f480effd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f481015e260 0x7f48100b49b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:35334/0 (socket says 172.21.15.16:35334) 2024-09-12T01:34:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f480f7fe640 1 -- 172.21.15.16:0/1961200241 learned_addr learned my addr 172.21.15.16:0/1961200241 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f480f7fe640 1 -- 172.21.15.16:0/1961200241 >> v2:172.21.15.33:3300/0 conn(0x7f481006baf0 msgr2=0x7f481006bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f480f7fe640 1 --2- 172.21.15.16:0/1961200241 >> v2:172.21.15.33:3300/0 conn(0x7f481006baf0 0x7f481006bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f480f7fe640 1 -- 172.21.15.16:0/1961200241 >> v1:172.21.15.33:6789/0 conn(0x7f481015e260 legacy=0x7f48100b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.731+0000 7f480f7fe640 1 -- 172.21.15.16:0/1961200241 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48100734a0 con 0x7f481006ff10 2024-09-12T01:34:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.732+0000 7f480f7fe640 1 --2- 172.21.15.16:0/1961200241 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f481006ff10 0x7f48100703a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4808009e50 tx=0x7f480802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccee98cb552bb006 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.732+0000 7f480dffb640 1 -- 172.21.15.16:0/1961200241 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808036070 con 0x7f481006ff10 2024-09-12T01:34:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.732+0000 7f480dffb640 1 -- 172.21.15.16:0/1961200241 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4808031600 con 0x7f481006ff10 2024-09-12T01:34:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.732+0000 7f480dffb640 1 -- 172.21.15.16:0/1961200241 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48080319d0 con 0x7f481006ff10 2024-09-12T01:34:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.733+0000 7f4815801640 1 -- 172.21.15.16:0/1961200241 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f481006ff10 msgr2=0x7f48100703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.733+0000 7f4815801640 1 --2- 172.21.15.16:0/1961200241 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f481006ff10 0x7f48100703a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4808009e50 tx=0x7f480802a710 comp rx=0 tx=0).stop 2024-09-12T01:34:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.733+0000 7f4815801640 1 -- 172.21.15.16:0/1961200241 shutdown_connections 2024-09-12T01:34:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.733+0000 7f4815801640 1 --2- 172.21.15.16:0/1961200241 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f481006ff10 0x7f48100703a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:00.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.733+0000 7f4815801640 1 --2- 172.21.15.16:0/1961200241 >> v2:172.21.15.33:3300/0 conn(0x7f481006baf0 0x7f481006bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:00.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.733+0000 7f4815801640 1 -- 172.21.15.16:0/1961200241 >> 172.21.15.16:0/1961200241 conn(0x7f48100d10b0 msgr2=0x7f48100d1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:00.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.733+0000 7f4815801640 1 -- 172.21.15.16:0/1961200241 shutdown_connections 2024-09-12T01:34:00.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.733+0000 7f4815801640 1 -- 172.21.15.16:0/1961200241 wait complete. 2024-09-12T01:34:00.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.734+0000 7f4815801640 1 Processor -- start 2024-09-12T01:34:00.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.734+0000 7f4815801640 1 -- start start 2024-09-12T01:34:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.735+0000 7f4815801640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f481006baf0 0x7f4810154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.735+0000 7f4815801640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481006ff10 0x7f481015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.735+0000 7f4815801640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f481015e260 0x7f481015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.735+0000 7f4815801640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48100b06d0 con 0x7f481006baf0 2024-09-12T01:34:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.735+0000 7f4815801640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4810073a60 con 0x7f481006ff10 2024-09-12T01:34:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.735+0000 7f4815801640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f4810072b70 con 0x7f481015e260 2024-09-12T01:34:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.735+0000 7f480effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f481006baf0 0x7f4810154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.735+0000 7f480effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f481006baf0 0x7f4810154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52700/0 (socket says 172.21.15.16:52700) 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.735+0000 7f480effd640 1 -- 172.21.15.16:0/3608373472 learned_addr learned my addr 172.21.15.16:0/3608373472 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.736+0000 7f480f7fe640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f481015e260 0x7f481015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.736+0000 7f480e7fc640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481006ff10 0x7f481015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.736+0000 7f480effd640 1 -- 172.21.15.16:0/3608373472 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f481015e260 msgr2=0x7f481015ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.736+0000 7f480effd640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f481015e260 0x7f481015ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.736+0000 7f480effd640 1 -- 172.21.15.16:0/3608373472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481006ff10 msgr2=0x7f481015b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.736+0000 7f480effd640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481006ff10 0x7f481015b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.736+0000 7f480effd640 1 -- 172.21.15.16:0/3608373472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481015bd90 con 0x7f481006baf0 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.736+0000 7f480f7fe640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f481015e260 0x7f481015ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.736+0000 7f480effd640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f481006baf0 0x7f4810154990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f480000c3c0 tx=0x7f480000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.736+0000 7f47f77fe640 1 -- 172.21.15.16:0/3608373472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48000074e0 con 0x7f481006baf0 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.736+0000 7f47f77fe640 1 -- 172.21.15.16:0/3608373472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4800007680 con 0x7f481006baf0 2024-09-12T01:34:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.736+0000 7f4815801640 1 -- 172.21.15.16:0/3608373472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481015c070 con 0x7f481006baf0 2024-09-12T01:34:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.737+0000 7f47f77fe640 1 -- 172.21.15.16:0/3608373472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800004a00 con 0x7f481006baf0 2024-09-12T01:34:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.737+0000 7f4815801640 1 -- 172.21.15.16:0/3608373472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481015c630 con 0x7f481006baf0 2024-09-12T01:34:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.738+0000 7f47f77fe640 1 -- 172.21.15.16:0/3608373472 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f480001a020 con 0x7f481006baf0 2024-09-12T01:34:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.738+0000 7f4815801640 1 -- 172.21.15.16:0/3608373472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48100b1ab0 con 0x7f481006baf0 2024-09-12T01:34:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.739+0000 7f47f77fe640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f47fc06be30 0x7f47fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.739+0000 7f47f77fe640 1 -- 172.21.15.16:0/3608373472 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5335+0+0 (secure 0 0 0) 0x7f480008a470 con 0x7f481006baf0 2024-09-12T01:34:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.739+0000 7f480e7fc640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f47fc06be30 0x7f47fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.739+0000 7f480e7fc640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f47fc06be30 0x7f47fc06e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4810160650 tx=0x7f4804008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.743+0000 7f47f77fe640 1 -- 172.21.15.16:0/3608373472 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4800058f90 con 0x7f481006baf0 2024-09-12T01:34:00.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:00.929+0000 7f4815801640 1 -- 172.21.15.16:0/3608373472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f48100b17d0 con 0x7f481006baf0 2024-09-12T01:34:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.535+0000 7f47f77fe640 1 -- 172.21.15.16:0/3608373472 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) ==== 97+0+0 (secure 0 0 0) 0x7f4800058890 con 0x7f481006baf0 2024-09-12T01:34:01.535 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-12T01:34:01.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.538+0000 7f4815801640 1 -- 172.21.15.16:0/3608373472 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f47fc06be30 msgr2=0x7f47fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:01.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.538+0000 7f4815801640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f47fc06be30 0x7f47fc06e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4810160650 tx=0x7f4804008040 comp rx=0 tx=0).stop 2024-09-12T01:34:01.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.538+0000 7f4815801640 1 -- 172.21.15.16:0/3608373472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f481006baf0 msgr2=0x7f4810154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.538+0000 7f4815801640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f481006baf0 0x7f4810154990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f480000c3c0 tx=0x7f480000c880 comp rx=0 tx=0).stop 2024-09-12T01:34:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.538+0000 7f4815801640 1 -- 172.21.15.16:0/3608373472 shutdown_connections 2024-09-12T01:34:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.538+0000 7f4815801640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f47fc06be30 0x7f47fc06e2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.538+0000 7f4815801640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f481015e260 0x7f481015ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.538+0000 7f4815801640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481006ff10 0x7f481015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.538+0000 7f4815801640 1 --2- 172.21.15.16:0/3608373472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f481006baf0 0x7f4810154990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.538+0000 7f4815801640 1 -- 172.21.15.16:0/3608373472 >> 172.21.15.16:0/3608373472 conn(0x7f48100d10b0 msgr2=0x7f48100adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.539+0000 7f4815801640 1 -- 172.21.15.16:0/3608373472 shutdown_connections 2024-09-12T01:34:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.539+0000 7f4815801640 1 -- 172.21.15.16:0/3608373472 wait complete. 2024-09-12T01:34:01.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T01:34:01.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.656+0000 7f54942f1640 1 Processor -- start 2024-09-12T01:34:01.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.656+0000 7f54942f1640 1 -- start start 2024-09-12T01:34:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f54942f1640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f548c15e800 0x7f548c0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f54942f1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f548c0cf910 0x7f548c06d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f54942f1640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f548c06dbd0 0x7f548c06e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f54942f1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f548c0cfd30 con 0x7f548c0cf910 2024-09-12T01:34:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f54942f1640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f548c15ebe0 con 0x7f548c15e800 2024-09-12T01:34:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f54942f1640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f548c0d42d0 con 0x7f548c06dbd0 2024-09-12T01:34:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f549188c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f548c0cf910 0x7f548c06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f549188c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f548c0cf910 0x7f548c06d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52704/0 (socket says 172.21.15.16:52704) 2024-09-12T01:34:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f549188c640 1 -- 172.21.15.16:0/2412904440 learned_addr learned my addr 172.21.15.16:0/2412904440 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f549188c640 1 -- 172.21.15.16:0/2412904440 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f548c06dbd0 msgr2=0x7f548c06e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f549188c640 1 --2- 172.21.15.16:0/2412904440 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f548c06dbd0 0x7f548c06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f549188c640 1 -- 172.21.15.16:0/2412904440 >> v2:172.21.15.33:3300/0 conn(0x7f548c15e800 msgr2=0x7f548c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f549208d640 1 --2- 172.21.15.16:0/2412904440 >> v2:172.21.15.33:3300/0 conn(0x7f548c15e800 0x7f548c0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f549188c640 1 --2- 172.21.15.16:0/2412904440 >> v2:172.21.15.33:3300/0 conn(0x7f548c15e800 0x7f548c0cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f549188c640 1 -- 172.21.15.16:0/2412904440 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c0d3ff0 con 0x7f548c0cf910 2024-09-12T01:34:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.657+0000 7f549208d640 1 --2- 172.21.15.16:0/2412904440 >> v2:172.21.15.33:3300/0 conn(0x7f548c15e800 0x7f548c0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:01.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.658+0000 7f549188c640 1 --2- 172.21.15.16:0/2412904440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0cf910 0x7f548c06d690 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f548c06c800 tx=0x7f548802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d159e70d890322 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.658+0000 7f549108b640 1 -- 172.21.15.16:0/2412904440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5488037070 con 0x7f548c0cf910 2024-09-12T01:34:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.658+0000 7f549108b640 1 -- 172.21.15.16:0/2412904440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5488032440 con 0x7f548c0cf910 2024-09-12T01:34:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.658+0000 7f549108b640 1 -- 172.21.15.16:0/2412904440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5488032810 con 0x7f548c0cf910 2024-09-12T01:34:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.659+0000 7f54942f1640 1 -- 172.21.15.16:0/2412904440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0cf910 msgr2=0x7f548c06d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.659+0000 7f54942f1640 1 --2- 172.21.15.16:0/2412904440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0cf910 0x7f548c06d690 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f548c06c800 tx=0x7f548802a1e0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.659+0000 7f54942f1640 1 -- 172.21.15.16:0/2412904440 shutdown_connections 2024-09-12T01:34:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.659+0000 7f54942f1640 1 --2- 172.21.15.16:0/2412904440 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f548c06dbd0 0x7f548c06e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.659+0000 7f54942f1640 1 --2- 172.21.15.16:0/2412904440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0cf910 0x7f548c06d690 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.659+0000 7f54942f1640 1 --2- 172.21.15.16:0/2412904440 >> v2:172.21.15.33:3300/0 conn(0x7f548c15e800 0x7f548c0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.659+0000 7f54942f1640 1 -- 172.21.15.16:0/2412904440 >> 172.21.15.16:0/2412904440 conn(0x7f548c09d750 msgr2=0x7f548c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.659+0000 7f54942f1640 1 -- 172.21.15.16:0/2412904440 shutdown_connections 2024-09-12T01:34:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.659+0000 7f54942f1640 1 -- 172.21.15.16:0/2412904440 wait complete. 2024-09-12T01:34:01.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.660+0000 7f54942f1640 1 Processor -- start 2024-09-12T01:34:01.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.660+0000 7f54942f1640 1 -- start start 2024-09-12T01:34:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f54942f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f548c06dbd0 0x7f548c14a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f54942f1640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f548c0cf910 0x7f548c1e6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f54942f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c15e800 0x7f548c15b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f54942f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f548c06d170 con 0x7f548c15e800 2024-09-12T01:34:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f54942f1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f548c06ce90 con 0x7f548c06dbd0 2024-09-12T01:34:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f54942f1640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f548c0d4e30 con 0x7f548c0cf910 2024-09-12T01:34:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f549288e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c15e800 0x7f548c15b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:01.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f549288e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c15e800 0x7f548c15b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52710/0 (socket says 172.21.15.16:52710) 2024-09-12T01:34:01.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f549288e640 1 -- 172.21.15.16:0/3270164409 learned_addr learned my addr 172.21.15.16:0/3270164409 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:01.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f549208d640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f548c06dbd0 0x7f548c14a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:01.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f549188c640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f548c0cf910 0x7f548c1e6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:01.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f549288e640 1 -- 172.21.15.16:0/3270164409 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f548c0cf910 msgr2=0x7f548c1e6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:01.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f549288e640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f548c0cf910 0x7f548c1e6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f549288e640 1 -- 172.21.15.16:0/3270164409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f548c06dbd0 msgr2=0x7f548c14a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f549288e640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f548c06dbd0 0x7f548c14a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.661+0000 7f549288e640 1 -- 172.21.15.16:0/3270164409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c0b9090 con 0x7f548c15e800 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.662+0000 7f549288e640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c15e800 0x7f548c15b060 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f548400b970 tx=0x7f548400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.662+0000 7f547effd640 1 -- 172.21.15.16:0/3270164409 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484019070 con 0x7f548c15e800 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.662+0000 7f547effd640 1 -- 172.21.15.16:0/3270164409 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f548400cb60 con 0x7f548c15e800 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.662+0000 7f54942f1640 1 -- 172.21.15.16:0/3270164409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548c0b9370 con 0x7f548c15e800 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.662+0000 7f547effd640 1 -- 172.21.15.16:0/3270164409 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484014440 con 0x7f548c15e800 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.662+0000 7f54942f1640 1 -- 172.21.15.16:0/3270164409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f548c0b9c10 con 0x7f548c15e800 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.663+0000 7f54942f1640 1 -- 172.21.15.16:0/3270164409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f548c0b9540 con 0x7f548c15e800 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.663+0000 7f547effd640 1 -- 172.21.15.16:0/3270164409 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f548401c070 con 0x7f548c15e800 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.664+0000 7f547effd640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f547006be30 0x7f547006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.664+0000 7f549208d640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f547006be30 0x7f547006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.664+0000 7f547effd640 1 -- 172.21.15.16:0/3270164409 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f548408a9c0 con 0x7f548c15e800 2024-09-12T01:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.664+0000 7f549208d640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f547006be30 0x7f547006e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f548000a990 tx=0x7f5480008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:01.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.668+0000 7f547effd640 1 -- 172.21.15.16:0/3270164409 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f548408ce20 con 0x7f548c15e800 2024-09-12T01:34:01.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.915+0000 7f54942f1640 1 -- 172.21.15.16:0/3270164409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f548c068880 con 0x7f548c15e800 2024-09-12T01:34:01.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.916+0000 7f547effd640 1 -- 172.21.15.16:0/3270164409 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f5484059530 con 0x7f548c15e800 2024-09-12T01:34:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:01.917 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:34:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-12T01:34:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-12T01:34:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCfROJmrQSeCxAA/0QxZw6X+WKa/poBP1nG3A==", 2024-09-12T01:34:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-12T01:34:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-12T01:34:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCfROJmuc/8FxAA/R1oc2RPaidINdOrcjez8g==", 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-12T01:34:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCfROJm6nrNEhAAJ4H4+3KC1mWOT/d/eUWtew==", 2024-09-12T01:34:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-12T01:34:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-12T01:34:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-12T01:34:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-12T01:34:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCfROJmXdhlHxAAgc1TWA2F7amP6+mYD7nPNw==", 2024-09-12T01:34:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-12T01:34:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-12T01:34:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-12T01:34:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-12T01:34:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQChROJmKyXsCBAAsqnBx3QHSaRg1xdqA3GAqA==", 2024-09-12T01:34:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-12T01:34:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-12T01:34:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-12T01:34:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCkROJmWcL9ChAAMF9aonFTkDzGd8LhRvZXTQ==", 2024-09-12T01:34:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-12T01:34:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-12T01:34:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCnROJmW0JwDRAAoxlXba7CaNVGw8dKQwatdQ==", 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCqROJmM7VWDxAANSE5pwGw3KJDpAOvvncT2w==", 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-12T01:34:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-12T01:34:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCwROJmMu8iGRAAB8dmNJfx9qY7hxMPCqPd0Q==", 2024-09-12T01:34:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-12T01:34:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-12T01:34:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCzROJmALnMGhAAAgYW8kCOG5jjw/mCSsI1vQ==", 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC2ROJmD1ncKxAAbewBA0yylXh538+uit6uWA==", 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-12T01:34:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC6ROJm9gXSABAABcCBjzd83OSzpRz1+k7Ggg==", 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-12T01:34:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCfROJmhMy2IxAAGVo/xGjEHxUMiQfBrw0mrQ==", 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCfROJmfwDBLBAABsu2b/R1NCaH5DT29W8JQg==", 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-12T01:34:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-12T01:34:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-12T01:34:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCfROJmxcxiKRAAAf2Zq/oLEIvWNh8dAKxOww==", 2024-09-12T01:34:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-12T01:34:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-12T01:34:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-12T01:34:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-12T01:34:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCfROJmsgRxMhAA2TuosZ1pMaOtZQPwcoZZeg==", 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCeROJmXkohKhAAOZLVWaUsaEk9nhB85vv4cg==", 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mds", 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDAROJmgNDsDBAAzx3n5xi/FTIR0JkaG/spnQ==", 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mds" 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mgr", 2024-09-12T01:34:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDAROJmFdrsDBAA4kyafZFSqTG3o7j5LZUnTw==", 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-osd", 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDAROJmqOHsDBAAagRIOr0eYNlDgGPar+eDBA==", 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-osd" 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd", 2024-09-12T01:34:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDAROJmPunsDBAAeGPx4K8JuHwvhsY84IlPdw==", 2024-09-12T01:34:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-12T01:34:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-12T01:34:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDAROJm8PDsDBAAUeDyPrl9zrN3pb88KwCAYA==", 2024-09-12T01:34:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-12T01:34:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rgw", 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDAROJm3vnsDBAAXRBj5FfK8P5nyBOilWIyRQ==", 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCfROJmFh9NCBAAfVJ6iVOOWFnk/XdcEXgxfg==", 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-12T01:34:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCfROJmG2gUBBAAdOEewBDrj7a7agWTq5zuWg==", 2024-09-12T01:34:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-12T01:34:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-12T01:34:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-12T01:34:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-12T01:34:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T01:34:01.933 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:34:01.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.918+0000 7f54942f1640 1 -- 172.21.15.16:0/3270164409 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f547006be30 msgr2=0x7f547006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:01.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.918+0000 7f54942f1640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f547006be30 0x7f547006e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f548000a990 tx=0x7f5480008040 comp rx=0 tx=0).stop 2024-09-12T01:34:01.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.918+0000 7f54942f1640 1 -- 172.21.15.16:0/3270164409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c15e800 msgr2=0x7f548c15b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:01.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.918+0000 7f54942f1640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c15e800 0x7f548c15b060 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f548400b970 tx=0x7f548400be30 comp rx=0 tx=0).stop 2024-09-12T01:34:01.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.918+0000 7f54942f1640 1 -- 172.21.15.16:0/3270164409 shutdown_connections 2024-09-12T01:34:01.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.918+0000 7f54942f1640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f547006be30 0x7f547006e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.918+0000 7f54942f1640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c15e800 0x7f548c15b060 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.918+0000 7f54942f1640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f548c0cf910 0x7f548c1e6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.918+0000 7f54942f1640 1 --2- 172.21.15.16:0/3270164409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f548c06dbd0 0x7f548c14a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:01.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.918+0000 7f54942f1640 1 -- 172.21.15.16:0/3270164409 >> 172.21.15.16:0/3270164409 conn(0x7f548c09d750 msgr2=0x7f548c0b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:01.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.918+0000 7f54942f1640 1 -- 172.21.15.16:0/3270164409 shutdown_connections 2024-09-12T01:34:01.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:01.918+0000 7f54942f1640 1 -- 172.21.15.16:0/3270164409 wait complete. 2024-09-12T01:34:01.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-12T01:34:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.038+0000 7f5f7930a640 1 Processor -- start 2024-09-12T01:34:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.039+0000 7f5f7930a640 1 -- start start 2024-09-12T01:34:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.039+0000 7f5f7930a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5f7415e280 0x7f5f740d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.039+0000 7f5f7930a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5f740d30b0 0x7f5f740ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.039+0000 7f5f7930a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f740d24d0 con 0x7f5f7415e660 2024-09-12T01:34:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.039+0000 7f5f7930a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f5f740b4c20 con 0x7f5f740d30b0 2024-09-12T01:34:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.039+0000 7f5f7930a640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f5f740b42f0 con 0x7f5f7415e280 2024-09-12T01:34:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.040+0000 7f5f72ffd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5f7415e280 0x7f5f740d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.040+0000 7f5f737fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5f740d30b0 0x7f5f740ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.040+0000 7f5f72ffd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5f7415e280 0x7f5f740d1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:54684/0 (socket says 172.21.15.16:54684) 2024-09-12T01:34:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.040+0000 7f5f737fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5f740d30b0 0x7f5f740ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45012/0 (socket says 172.21.15.16:45012) 2024-09-12T01:34:02.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.040+0000 7f5f72ffd640 1 -- 172.21.15.16:0/2183940987 learned_addr learned my addr 172.21.15.16:0/2183940987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:02.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.040+0000 7f5f727fc640 1 --1- 172.21.15.16:0/2183940987 >> v1:172.21.15.33:6789/0 conn(0x7f5f7415e660 0x7f5f740d29a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:35338/0 (socket says 172.21.15.16:35338) 2024-09-12T01:34:02.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.040+0000 7f5f72ffd640 1 -- 172.21.15.16:0/2183940987 >> v2:172.21.15.33:3300/0 conn(0x7f5f740d30b0 msgr2=0x7f5f740ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:02.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.040+0000 7f5f72ffd640 1 --2- 172.21.15.16:0/2183940987 >> v2:172.21.15.33:3300/0 conn(0x7f5f740d30b0 0x7f5f740ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.040+0000 7f5f72ffd640 1 -- 172.21.15.16:0/2183940987 >> v1:172.21.15.33:6789/0 conn(0x7f5f7415e660 legacy=0x7f5f740d29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.040+0000 7f5f72ffd640 1 -- 172.21.15.16:0/2183940987 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f740b45d0 con 0x7f5f7415e280 2024-09-12T01:34:02.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.040+0000 7f5f737fe640 1 --2- 172.21.15.16:0/2183940987 >> v2:172.21.15.33:3300/0 conn(0x7f5f740d30b0 0x7f5f740ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.041+0000 7f5f72ffd640 1 --2- 172.21.15.16:0/2183940987 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5f7415e280 0x7f5f740d1f90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009870 tx=0x7f5f6402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d4c29dff17427a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.041+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2183940987 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64037070 con 0x7f5f7415e280 2024-09-12T01:34:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.041+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2183940987 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f64032440 con 0x7f5f7415e280 2024-09-12T01:34:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.041+0000 7f5f71ffb640 1 -- 172.21.15.16:0/2183940987 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64032810 con 0x7f5f7415e280 2024-09-12T01:34:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.042+0000 7f5f7930a640 1 -- 172.21.15.16:0/2183940987 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5f7415e280 msgr2=0x7f5f740d1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.042+0000 7f5f7930a640 1 --2- 172.21.15.16:0/2183940987 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5f7415e280 0x7f5f740d1f90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009870 tx=0x7f5f6402a270 comp rx=0 tx=0).stop 2024-09-12T01:34:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.042+0000 7f5f7930a640 1 -- 172.21.15.16:0/2183940987 shutdown_connections 2024-09-12T01:34:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.042+0000 7f5f7930a640 1 --2- 172.21.15.16:0/2183940987 >> v2:172.21.15.33:3300/0 conn(0x7f5f740d30b0 0x7f5f740ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.042+0000 7f5f7930a640 1 --2- 172.21.15.16:0/2183940987 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5f7415e280 0x7f5f740d1f90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.042+0000 7f5f7930a640 1 -- 172.21.15.16:0/2183940987 >> 172.21.15.16:0/2183940987 conn(0x7f5f7408ddf0 msgr2=0x7f5f7409d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.042+0000 7f5f7930a640 1 -- 172.21.15.16:0/2183940987 shutdown_connections 2024-09-12T01:34:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.042+0000 7f5f7930a640 1 -- 172.21.15.16:0/2183940987 wait complete. 2024-09-12T01:34:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.043+0000 7f5f7930a640 1 Processor -- start 2024-09-12T01:34:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.043+0000 7f5f7930a640 1 -- start start 2024-09-12T01:34:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f7930a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f740d30b0 0x7f5f741f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f7930a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f7415e280 0x7f5f7406d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f7930a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5f7415e660 0x7f5f7415f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f7930a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f740b5230 con 0x7f5f7415e280 2024-09-12T01:34:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f7930a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f740b50b0 con 0x7f5f740d30b0 2024-09-12T01:34:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f7930a640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f5f740b3a30 con 0x7f5f7415e660 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f727fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f7415e280 0x7f5f7406d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f727fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f7415e280 0x7f5f7406d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52712/0 (socket says 172.21.15.16:52712) 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f727fc640 1 -- 172.21.15.16:0/2683628066 learned_addr learned my addr 172.21.15.16:0/2683628066 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f72ffd640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f740d30b0 0x7f5f741f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f737fe640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5f7415e660 0x7f5f7415f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f727fc640 1 -- 172.21.15.16:0/2683628066 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5f7415e660 msgr2=0x7f5f7415f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f727fc640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5f7415e660 0x7f5f7415f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f727fc640 1 -- 172.21.15.16:0/2683628066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f740d30b0 msgr2=0x7f5f741f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f727fc640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f740d30b0 0x7f5f741f75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.044+0000 7f5f727fc640 1 -- 172.21.15.16:0/2683628066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f741592b0 con 0x7f5f7415e280 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.045+0000 7f5f727fc640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f7415e280 0x7f5f7406d020 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800c3f0 tx=0x7f5f6800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.045+0000 7f5f5b7fe640 1 -- 172.21.15.16:0/2683628066 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f680074e0 con 0x7f5f7415e280 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.045+0000 7f5f5b7fe640 1 -- 172.21.15.16:0/2683628066 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f68007680 con 0x7f5f7415e280 2024-09-12T01:34:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.045+0000 7f5f7930a640 1 -- 172.21.15.16:0/2683628066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f741f7ca0 con 0x7f5f7415e280 2024-09-12T01:34:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.045+0000 7f5f5b7fe640 1 -- 172.21.15.16:0/2683628066 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68004a00 con 0x7f5f7415e280 2024-09-12T01:34:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.045+0000 7f5f7930a640 1 -- 172.21.15.16:0/2683628066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f741f8260 con 0x7f5f7415e280 2024-09-12T01:34:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.046+0000 7f5f7930a640 1 -- 172.21.15.16:0/2683628066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f740d1f90 con 0x7f5f7415e280 2024-09-12T01:34:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.046+0000 7f5f5b7fe640 1 -- 172.21.15.16:0/2683628066 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f6801a020 con 0x7f5f7415e280 2024-09-12T01:34:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.047+0000 7f5f5b7fe640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5f6006be30 0x7f5f6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.047+0000 7f5f5b7fe640 1 -- 172.21.15.16:0/2683628066 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f5f6808a350 con 0x7f5f7415e280 2024-09-12T01:34:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.047+0000 7f5f72ffd640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5f6006be30 0x7f5f6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.047+0000 7f5f72ffd640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5f6006be30 0x7f5f6006e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5f64034000 tx=0x7f5f6402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.051+0000 7f5f5b7fe640 1 -- 172.21.15.16:0/2683628066 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5f740d24d0 con 0x7f5f7415e280 2024-09-12T01:34:02.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.298+0000 7f5f7930a640 1 -- 172.21.15.16:0/2683628066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f5f740d1970 con 0x7f5f7415e280 2024-09-12T01:34:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.300+0000 7f5f5b7fe640 1 -- 172.21.15.16:0/2683628066 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f5f68058e40 con 0x7f5f7415e280 2024-09-12T01:34:02.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.302+0000 7f5f7930a640 1 -- 172.21.15.16:0/2683628066 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5f6006be30 msgr2=0x7f5f6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.302+0000 7f5f7930a640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5f6006be30 0x7f5f6006e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5f64034000 tx=0x7f5f6402e000 comp rx=0 tx=0).stop 2024-09-12T01:34:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.302+0000 7f5f7930a640 1 -- 172.21.15.16:0/2683628066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f7415e280 msgr2=0x7f5f7406d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.302+0000 7f5f7930a640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f7415e280 0x7f5f7406d020 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800c3f0 tx=0x7f5f6800c7f0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.302+0000 7f5f7930a640 1 -- 172.21.15.16:0/2683628066 shutdown_connections 2024-09-12T01:34:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.302+0000 7f5f7930a640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5f6006be30 0x7f5f6006e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.302+0000 7f5f7930a640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5f7415e660 0x7f5f7415f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.302+0000 7f5f7930a640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f7415e280 0x7f5f7406d020 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.302+0000 7f5f7930a640 1 --2- 172.21.15.16:0/2683628066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f740d30b0 0x7f5f741f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.302+0000 7f5f7930a640 1 -- 172.21.15.16:0/2683628066 >> 172.21.15.16:0/2683628066 conn(0x7f5f7408ddf0 msgr2=0x7f5f740d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.302+0000 7f5f7930a640 1 -- 172.21.15.16:0/2683628066 shutdown_connections 2024-09-12T01:34:02.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.302+0000 7f5f7930a640 1 -- 172.21.15.16:0/2683628066 wait complete. 2024-09-12T01:34:02.313 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-12T01:34:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.416+0000 7ff7b34d0640 1 Processor -- start 2024-09-12T01:34:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.416+0000 7ff7b34d0640 1 -- start start 2024-09-12T01:34:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.417+0000 7ff7b34d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7ac0b86d0 0x7ff7ac169070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.417+0000 7ff7b34d0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7ac161dc0 con 0x7ff7ac1619e0 2024-09-12T01:34:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.417+0000 7ff7b34d0640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7ac0d3310 con 0x7ff7ac0d0cc0 2024-09-12T01:34:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.417+0000 7ff7b34d0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff7ac0d3f20 con 0x7ff7ac0b86d0 2024-09-12T01:34:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.417+0000 7ff7b1a6d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7ac0b86d0 0x7ff7ac169070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.417+0000 7ff7b126c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff7ac1619e0 0x7ff7ac0d2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48308/0 (socket says 172.21.15.16:48308) 2024-09-12T01:34:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.417+0000 7ff7b126c640 1 -- 172.21.15.16:0/1485527099 learned_addr learned my addr 172.21.15.16:0/1485527099 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.417+0000 7ff7b1a6d640 1 -- 172.21.15.16:0/1485527099 >> v1:172.21.15.33:6789/0 conn(0x7ff7ac0d0cc0 legacy=0x7ff7ac0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.417+0000 7ff7b1a6d640 1 -- 172.21.15.16:0/1485527099 >> v1:172.21.15.16:6789/0 conn(0x7ff7ac1619e0 legacy=0x7ff7ac0d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.417+0000 7ff7b1a6d640 1 -- 172.21.15.16:0/1485527099 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac0d35f0 con 0x7ff7ac0b86d0 2024-09-12T01:34:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.418+0000 7ff7b1a6d640 1 --2- 172.21.15.16:0/1485527099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7ac0b86d0 0x7ff7ac169070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009d90 tx=0x7ff7a402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b86ff6809b869f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.418+0000 7ff7a3fff640 1 -- 172.21.15.16:0/1485527099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4007740 con 0x7ff7ac0b86d0 2024-09-12T01:34:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.418+0000 7ff7a3fff640 1 -- 172.21.15.16:0/1485527099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a40078e0 con 0x7ff7ac0b86d0 2024-09-12T01:34:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.418+0000 7ff7a3fff640 1 -- 172.21.15.16:0/1485527099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4007cb0 con 0x7ff7ac0b86d0 2024-09-12T01:34:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.419+0000 7ff7b34d0640 1 -- 172.21.15.16:0/1485527099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7ac0b86d0 msgr2=0x7ff7ac169070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.419+0000 7ff7b34d0640 1 --2- 172.21.15.16:0/1485527099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7ac0b86d0 0x7ff7ac169070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009d90 tx=0x7ff7a402a6a0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.419+0000 7ff7b34d0640 1 -- 172.21.15.16:0/1485527099 shutdown_connections 2024-09-12T01:34:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.419+0000 7ff7b34d0640 1 --2- 172.21.15.16:0/1485527099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7ac0b86d0 0x7ff7ac169070 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.419+0000 7ff7b34d0640 1 -- 172.21.15.16:0/1485527099 >> 172.21.15.16:0/1485527099 conn(0x7ff7ac06d1c0 msgr2=0x7ff7ac073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.419+0000 7ff7b34d0640 1 -- 172.21.15.16:0/1485527099 shutdown_connections 2024-09-12T01:34:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.419+0000 7ff7b34d0640 1 -- 172.21.15.16:0/1485527099 wait complete. 2024-09-12T01:34:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.420+0000 7ff7b34d0640 1 Processor -- start 2024-09-12T01:34:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.420+0000 7ff7b34d0640 1 -- start start 2024-09-12T01:34:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b34d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7ac0b86d0 0x7ff7ac1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b34d0640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7ac0d0cc0 0x7ff7ac0b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b34d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7ac1619e0 0x7ff7ac0b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b34d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac0d2250 con 0x7ff7ac1619e0 2024-09-12T01:34:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b34d0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac0d3bb0 con 0x7ff7ac0b86d0 2024-09-12T01:34:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b34d0640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ac0d2f60 con 0x7ff7ac0d0cc0 2024-09-12T01:34:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b1a6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7ac1619e0 0x7ff7ac0b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b1a6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7ac1619e0 0x7ff7ac0b3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52730/0 (socket says 172.21.15.16:52730) 2024-09-12T01:34:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b1a6d640 1 -- 172.21.15.16:0/2104771846 learned_addr learned my addr 172.21.15.16:0/2104771846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b126c640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7ac0b86d0 0x7ff7ac1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b0a6b640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7ac0d0cc0 0x7ff7ac0b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b1a6d640 1 -- 172.21.15.16:0/2104771846 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7ac0d0cc0 msgr2=0x7ff7ac0b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b1a6d640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7ac0d0cc0 0x7ff7ac0b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b1a6d640 1 -- 172.21.15.16:0/2104771846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7ac0b86d0 msgr2=0x7ff7ac1594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b1a6d640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7ac0b86d0 0x7ff7ac1594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b1a6d640 1 -- 172.21.15.16:0/2104771846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac15b0e0 con 0x7ff7ac1619e0 2024-09-12T01:34:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7b1a6d640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7ac1619e0 0x7ff7ac0b3540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009eb0 tx=0x7ff7a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.421+0000 7ff7a1ffb640 1 -- 172.21.15.16:0/2104771846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a403f070 con 0x7ff7ac1619e0 2024-09-12T01:34:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.422+0000 7ff7a1ffb640 1 -- 172.21.15.16:0/2104771846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a4031c90 con 0x7ff7ac1619e0 2024-09-12T01:34:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.422+0000 7ff7b34d0640 1 -- 172.21.15.16:0/2104771846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ac15b380 con 0x7ff7ac1619e0 2024-09-12T01:34:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.422+0000 7ff7a1ffb640 1 -- 172.21.15.16:0/2104771846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4043630 con 0x7ff7ac1619e0 2024-09-12T01:34:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.422+0000 7ff7b34d0640 1 -- 172.21.15.16:0/2104771846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ac15b940 con 0x7ff7ac1619e0 2024-09-12T01:34:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.423+0000 7ff7b34d0640 1 -- 172.21.15.16:0/2104771846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7ac0c7ae0 con 0x7ff7ac1619e0 2024-09-12T01:34:02.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.423+0000 7ff7a1ffb640 1 -- 172.21.15.16:0/2104771846 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff7a403b410 con 0x7ff7ac1619e0 2024-09-12T01:34:02.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.424+0000 7ff7a1ffb640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff79006be30 0x7ff79006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.424+0000 7ff7a1ffb640 1 -- 172.21.15.16:0/2104771846 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7ff7a40ac160 con 0x7ff7ac1619e0 2024-09-12T01:34:02.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.424+0000 7ff7b126c640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff79006be30 0x7ff79006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.424+0000 7ff7b126c640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff79006be30 0x7ff79006e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8007fd0 tx=0x7ff7a8007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.428+0000 7ff7a1ffb640 1 -- 172.21.15.16:0/2104771846 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff7a407a6a0 con 0x7ff7ac1619e0 2024-09-12T01:34:02.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.678+0000 7ff7b34d0640 1 -- 172.21.15.16:0/2104771846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7ff7ac157eb0 con 0x7ff7ac1619e0 2024-09-12T01:34:02.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.680+0000 7ff7a1ffb640 1 -- 172.21.15.16:0/2104771846 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7ff7a407e9a0 con 0x7ff7ac1619e0 2024-09-12T01:34:02.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.682+0000 7ff7b34d0640 1 -- 172.21.15.16:0/2104771846 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff79006be30 msgr2=0x7ff79006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.682+0000 7ff7b34d0640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff79006be30 0x7ff79006e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8007fd0 tx=0x7ff7a8007e80 comp rx=0 tx=0).stop 2024-09-12T01:34:02.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.682+0000 7ff7b34d0640 1 -- 172.21.15.16:0/2104771846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7ac1619e0 msgr2=0x7ff7ac0b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.682+0000 7ff7b34d0640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7ac1619e0 0x7ff7ac0b3540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009eb0 tx=0x7ff7a4004a40 comp rx=0 tx=0).stop 2024-09-12T01:34:02.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.682+0000 7ff7b34d0640 1 -- 172.21.15.16:0/2104771846 shutdown_connections 2024-09-12T01:34:02.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.682+0000 7ff7b34d0640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff79006be30 0x7ff79006e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.682+0000 7ff7b34d0640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7ac1619e0 0x7ff7ac0b3540 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.682+0000 7ff7b34d0640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff7ac0d0cc0 0x7ff7ac0b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.682+0000 7ff7b34d0640 1 --2- 172.21.15.16:0/2104771846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7ac0b86d0 0x7ff7ac1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.682+0000 7ff7b34d0640 1 -- 172.21.15.16:0/2104771846 >> 172.21.15.16:0/2104771846 conn(0x7ff7ac06d1c0 msgr2=0x7ff7ac0b96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:02.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.682+0000 7ff7b34d0640 1 -- 172.21.15.16:0/2104771846 shutdown_connections 2024-09-12T01:34:02.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.682+0000 7ff7b34d0640 1 -- 172.21.15.16:0/2104771846 wait complete. 2024-09-12T01:34:02.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-12T01:34:02.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.796+0000 7f14c7b1a640 1 Processor -- start 2024-09-12T01:34:02.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.797+0000 7f14c7b1a640 1 -- start start 2024-09-12T01:34:02.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.797+0000 7f14c7b1a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f14c006dd90 0x7f14c00acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.797+0000 7f14c7b1a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c006f530 con 0x7f14c015e8d0 2024-09-12T01:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.797+0000 7f14c7b1a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c00cbbb0 con 0x7f14c00cb580 2024-09-12T01:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.797+0000 7f14c7b1a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f14c00cbe90 con 0x7f14c006dd90 2024-09-12T01:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c58b6640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f14c015e8d0 0x7f14c00cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48314/0 (socket says 172.21.15.16:48314) 2024-09-12T01:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c58b6640 1 -- 172.21.15.16:0/2532041451 learned_addr learned my addr 172.21.15.16:0/2532041451 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1793770597 0 0) 0x7f14c006f530 con 0x7f14c015e8d0 2024-09-12T01:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14ac003520 con 0x7f14c015e8d0 2024-09-12T01:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c50b5640 1 --2- 172.21.15.16:0/2532041451 >> v2:172.21.15.33:3300/0 conn(0x7f14c006dd90 0x7f14c00acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2160616111 0 0) 0x7f14ac003520 con 0x7f14c015e8d0 2024-09-12T01:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 >> v2:172.21.15.33:3300/0 conn(0x7f14c006dd90 msgr2=0x7f14c00acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 --2- 172.21.15.16:0/2532041451 >> v2:172.21.15.33:3300/0 conn(0x7f14c006dd90 0x7f14c00acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 >> v1:172.21.15.33:6789/0 conn(0x7f14c00cb580 legacy=0x7f14c00ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c00ca5f0 con 0x7f14c015e8d0 2024-09-12T01:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f14b4001670 con 0x7f14c015e8d0 2024-09-12T01:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 >> v1:172.21.15.16:6789/0 conn(0x7f14c015e8d0 legacy=0x7f14c00cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 --2- 172.21.15.16:0/2532041451 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f14ac003a20 0x7f14ac003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 --2- 172.21.15.16:0/2532041451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac004800 0x7f14ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 --2- 172.21.15.16:0/2532041451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14ac0051c0 0x7f14ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14c00cbe90 con 0x7f14ac004800 2024-09-12T01:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac00c160 con 0x7f14ac0051c0 2024-09-12T01:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f14ac00c360 con 0x7f14ac003a20 2024-09-12T01:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c50b5640 1 --2- 172.21.15.16:0/2532041451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac004800 0x7f14ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c58b6640 1 --2- 172.21.15.16:0/2532041451 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f14ac003a20 0x7f14ac003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.798+0000 7f14c60b7640 1 --2- 172.21.15.16:0/2532041451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14ac0051c0 0x7f14ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.799+0000 7f14c50b5640 1 -- 172.21.15.16:0/2532041451 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f14ac003a20 msgr2=0x7f14ac003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.799+0000 7f14c50b5640 1 --2- 172.21.15.16:0/2532041451 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f14ac003a20 0x7f14ac003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.799+0000 7f14c50b5640 1 -- 172.21.15.16:0/2532041451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14ac0051c0 msgr2=0x7f14ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.799+0000 7f14c50b5640 1 --2- 172.21.15.16:0/2532041451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14ac0051c0 0x7f14ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.799+0000 7f14c50b5640 1 -- 172.21.15.16:0/2532041451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac003520 con 0x7f14ac004800 2024-09-12T01:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.799+0000 7f14c60b7640 1 -- 172.21.15.16:0/2532041451 reap_dead start 2024-09-12T01:34:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.799+0000 7f14c50b5640 1 --2- 172.21.15.16:0/2532041451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac004800 0x7f14ac004c80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009ac0 tx=0x7f14ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7482102abbe06d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.800+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc036070 con 0x7f14ac004800 2024-09-12T01:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.800+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14bc031400 con 0x7f14ac004800 2024-09-12T01:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.800+0000 7f14c48b4640 1 -- 172.21.15.16:0/2532041451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc009690 con 0x7f14ac004800 2024-09-12T01:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.800+0000 7f14c7b1a640 1 -- 172.21.15.16:0/2532041451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac004800 msgr2=0x7f14ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.800+0000 7f14c7b1a640 1 --2- 172.21.15.16:0/2532041451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac004800 0x7f14ac004c80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009ac0 tx=0x7f14ac005830 comp rx=0 tx=0).stop 2024-09-12T01:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.800+0000 7f14c7b1a640 1 -- 172.21.15.16:0/2532041451 shutdown_connections 2024-09-12T01:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.800+0000 7f14c7b1a640 1 --2- 172.21.15.16:0/2532041451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac004800 0x7f14ac004c80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.800+0000 7f14c7b1a640 1 -- 172.21.15.16:0/2532041451 >> 172.21.15.16:0/2532041451 conn(0x7f14c00cffa0 msgr2=0x7f14c00d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.800+0000 7f14c7b1a640 1 -- 172.21.15.16:0/2532041451 shutdown_connections 2024-09-12T01:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.800+0000 7f14c7b1a640 1 -- 172.21.15.16:0/2532041451 wait complete. 2024-09-12T01:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.801+0000 7f14c7b1a640 1 Processor -- start 2024-09-12T01:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.802+0000 7f14c7b1a640 1 -- start start 2024-09-12T01:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.802+0000 7f14c7b1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac004800 0x7f14c01594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.802+0000 7f14c7b1a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f14c0156f30 0x7f14c01f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.802+0000 7f14c7b1a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c01f2050 0x7f14c01f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.802+0000 7f14c7b1a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14c00acdd0 con 0x7f14ac004800 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.802+0000 7f14c7b1a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14c00cabb0 con 0x7f14c01f2050 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.802+0000 7f14c7b1a640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f14c00acf50 con 0x7f14c0156f30 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.802+0000 7f14c58b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac004800 0x7f14c01594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.802+0000 7f14c58b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac004800 0x7f14c01594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52744/0 (socket says 172.21.15.16:52744) 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.802+0000 7f14c58b6640 1 -- 172.21.15.16:0/1895849670 learned_addr learned my addr 172.21.15.16:0/1895849670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.803+0000 7f14c60b7640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c01f2050 0x7f14c01f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.803+0000 7f14c50b5640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f14c0156f30 0x7f14c01f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.803+0000 7f14c58b6640 1 -- 172.21.15.16:0/1895849670 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f14c0156f30 msgr2=0x7f14c01f1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.803+0000 7f14c58b6640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f14c0156f30 0x7f14c01f1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.803+0000 7f14c58b6640 1 -- 172.21.15.16:0/1895849670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c01f2050 msgr2=0x7f14c01f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.803+0000 7f14c58b6640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c01f2050 0x7f14c01f24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.803+0000 7f14c58b6640 1 -- 172.21.15.16:0/1895849670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c01f2a10 con 0x7f14ac004800 2024-09-12T01:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.803+0000 7f14c58b6640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac004800 0x7f14c01594b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f14b40072c0 tx=0x7f14b4007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.803+0000 7f14b27fc640 1 -- 172.21.15.16:0/1895849670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b400f8e0 con 0x7f14ac004800 2024-09-12T01:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.803+0000 7f14b27fc640 1 -- 172.21.15.16:0/1895849670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b4018070 con 0x7f14ac004800 2024-09-12T01:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.803+0000 7f14c7b1a640 1 -- 172.21.15.16:0/1895849670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c0159a90 con 0x7f14ac004800 2024-09-12T01:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.803+0000 7f14b27fc640 1 -- 172.21.15.16:0/1895849670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b400fca0 con 0x7f14ac004800 2024-09-12T01:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.803+0000 7f14c7b1a640 1 -- 172.21.15.16:0/1895849670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c00afb90 con 0x7f14ac004800 2024-09-12T01:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.804+0000 7f14c7b1a640 1 -- 172.21.15.16:0/1895849670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c015dd60 con 0x7f14ac004800 2024-09-12T01:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.805+0000 7f14b27fc640 1 -- 172.21.15.16:0/1895849670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f14b401d460 con 0x7f14ac004800 2024-09-12T01:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.805+0000 7f14b27fc640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f14a406be60 0x7f14a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.805+0000 7f14c50b5640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f14a406be60 0x7f14a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.805+0000 7f14b27fc640 1 -- 172.21.15.16:0/1895849670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f14b408aab0 con 0x7f14ac004800 2024-09-12T01:34:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.806+0000 7f14c50b5640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f14a406be60 0x7f14a406e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14bc033000 tx=0x7f14bc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:02.810+0000 7f14b27fc640 1 -- 172.21.15.16:0/1895849670 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f14b4059060 con 0x7f14ac004800 2024-09-12T01:34:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.058+0000 7f14c7b1a640 1 -- 172.21.15.16:0/1895849670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f14c0159fb0 con 0x7f14ac004800 2024-09-12T01:34:03.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.060+0000 7f14b27fc640 1 -- 172.21.15.16:0/1895849670 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f14b405d360 con 0x7f14ac004800 2024-09-12T01:34:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.062+0000 7f14c7b1a640 1 -- 172.21.15.16:0/1895849670 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f14a406be60 msgr2=0x7f14a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.062+0000 7f14c7b1a640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f14a406be60 0x7f14a406e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14bc033000 tx=0x7f14bc008030 comp rx=0 tx=0).stop 2024-09-12T01:34:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.062+0000 7f14c7b1a640 1 -- 172.21.15.16:0/1895849670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac004800 msgr2=0x7f14c01594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.062+0000 7f14c7b1a640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac004800 0x7f14c01594b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f14b40072c0 tx=0x7f14b4007780 comp rx=0 tx=0).stop 2024-09-12T01:34:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.062+0000 7f14c7b1a640 1 -- 172.21.15.16:0/1895849670 shutdown_connections 2024-09-12T01:34:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.062+0000 7f14c7b1a640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f14a406be60 0x7f14a406e320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.062+0000 7f14c7b1a640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c01f2050 0x7f14c01f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.062+0000 7f14c7b1a640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f14c0156f30 0x7f14c01f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.062+0000 7f14c7b1a640 1 --2- 172.21.15.16:0/1895849670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac004800 0x7f14c01594b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.062+0000 7f14c7b1a640 1 -- 172.21.15.16:0/1895849670 >> 172.21.15.16:0/1895849670 conn(0x7f14c00cffa0 msgr2=0x7f14c015ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.062+0000 7f14c7b1a640 1 -- 172.21.15.16:0/1895849670 shutdown_connections 2024-09-12T01:34:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.062+0000 7f14c7b1a640 1 -- 172.21.15.16:0/1895849670 wait complete. 2024-09-12T01:34:03.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-12T01:34:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.176+0000 7fc6166d0640 1 Processor -- start 2024-09-12T01:34:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.176+0000 7fc6166d0640 1 -- start start 2024-09-12T01:34:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc6166d0640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc61015e660 0x7fc6100d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc6166d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6100d4d60 0x7fc6100d4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc6166d0640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc610162020 0x7fc6100b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc6166d0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc6100d2020 con 0x7fc6100d4d60 2024-09-12T01:34:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc6166d0640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fc610162500 con 0x7fc61015e660 2024-09-12T01:34:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc6166d0640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fc6101627e0 con 0x7fc610162020 2024-09-12T01:34:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc607fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6100d4d60 0x7fc6100d4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc607fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6100d4d60 0x7fc6100d4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52758/0 (socket says 172.21.15.16:52758) 2024-09-12T01:34:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc607fff640 1 -- 172.21.15.16:0/2896925028 learned_addr learned my addr 172.21.15.16:0/2896925028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc614c6d640 1 --2- 172.21.15.16:0/2896925028 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc610162020 0x7fc6100b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc60ffff640 1 --2- 172.21.15.16:0/2896925028 >> v2:172.21.15.33:3300/0 conn(0x7fc61015e660 0x7fc6100d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc607fff640 1 -- 172.21.15.16:0/2896925028 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc610162020 msgr2=0x7fc6100b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc607fff640 1 --2- 172.21.15.16:0/2896925028 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc610162020 0x7fc6100b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc607fff640 1 -- 172.21.15.16:0/2896925028 >> v2:172.21.15.33:3300/0 conn(0x7fc61015e660 msgr2=0x7fc6100d4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:03.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc607fff640 1 --2- 172.21.15.16:0/2896925028 >> v2:172.21.15.33:3300/0 conn(0x7fc61015e660 0x7fc6100d4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.177+0000 7fc607fff640 1 -- 172.21.15.16:0/2896925028 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc610162ac0 con 0x7fc6100d4d60 2024-09-12T01:34:03.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.178+0000 7fc607fff640 1 --2- 172.21.15.16:0/2896925028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6100d4d60 0x7fc6100d4640 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc61006c250 tx=0x7fc5f802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78084f4ff2fea07e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.178+0000 7fc60f7fe640 1 -- 172.21.15.16:0/2896925028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8037070 con 0x7fc6100d4d60 2024-09-12T01:34:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.178+0000 7fc60f7fe640 1 -- 172.21.15.16:0/2896925028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f8032440 con 0x7fc6100d4d60 2024-09-12T01:34:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.178+0000 7fc60f7fe640 1 -- 172.21.15.16:0/2896925028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8032810 con 0x7fc6100d4d60 2024-09-12T01:34:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.179+0000 7fc6166d0640 1 -- 172.21.15.16:0/2896925028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6100d4d60 msgr2=0x7fc6100d4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.179+0000 7fc6166d0640 1 --2- 172.21.15.16:0/2896925028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6100d4d60 0x7fc6100d4640 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc61006c250 tx=0x7fc5f802a140 comp rx=0 tx=0).stop 2024-09-12T01:34:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.179+0000 7fc6166d0640 1 -- 172.21.15.16:0/2896925028 shutdown_connections 2024-09-12T01:34:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.179+0000 7fc6166d0640 1 --2- 172.21.15.16:0/2896925028 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc610162020 0x7fc6100b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.179+0000 7fc6166d0640 1 --2- 172.21.15.16:0/2896925028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6100d4d60 0x7fc6100d4640 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.179+0000 7fc6166d0640 1 --2- 172.21.15.16:0/2896925028 >> v2:172.21.15.33:3300/0 conn(0x7fc61015e660 0x7fc6100d4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.179+0000 7fc6166d0640 1 -- 172.21.15.16:0/2896925028 >> 172.21.15.16:0/2896925028 conn(0x7fc61008de10 msgr2=0x7fc61009d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.179+0000 7fc6166d0640 1 -- 172.21.15.16:0/2896925028 shutdown_connections 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.179+0000 7fc6166d0640 1 -- 172.21.15.16:0/2896925028 wait complete. 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.180+0000 7fc6166d0640 1 Processor -- start 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.180+0000 7fc6166d0640 1 -- start start 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc6166d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6100d4d60 0x7fc610150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc6166d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc61015e660 0x7fc61015fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc6166d0640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc610162020 0x7fc6100b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc6166d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6100d0fb0 con 0x7fc6100d4d60 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc6166d0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc6100d0e30 con 0x7fc61015e660 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc6166d0640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fc6100d1130 con 0x7fc610162020 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc60ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6100d4d60 0x7fc610150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc60ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6100d4d60 0x7fc610150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52772/0 (socket says 172.21.15.16:52772) 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc60ffff640 1 -- 172.21.15.16:0/3962401776 learned_addr learned my addr 172.21.15.16:0/3962401776 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc614c6d640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc610162020 0x7fc6100b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc607fff640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc61015e660 0x7fc61015fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc60ffff640 1 -- 172.21.15.16:0/3962401776 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc610162020 msgr2=0x7fc6100b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc60ffff640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc610162020 0x7fc6100b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc60ffff640 1 -- 172.21.15.16:0/3962401776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc61015e660 msgr2=0x7fc61015fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc60ffff640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc61015e660 0x7fc61015fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc60ffff640 1 -- 172.21.15.16:0/3962401776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6101605a0 con 0x7fc6100d4d60 2024-09-12T01:34:03.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc607fff640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc61015e660 0x7fc61015fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:03.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc60ffff640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6100d4d60 0x7fc610150760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc60000d830 tx=0x7fc60000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc60d7fa640 1 -- 172.21.15.16:0/3962401776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600018020 con 0x7fc6100d4d60 2024-09-12T01:34:03.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc60d7fa640 1 -- 172.21.15.16:0/3962401776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6000096a0 con 0x7fc6100d4d60 2024-09-12T01:34:03.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc60d7fa640 1 -- 172.21.15.16:0/3962401776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600009a70 con 0x7fc6100d4d60 2024-09-12T01:34:03.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc6166d0640 1 -- 172.21.15.16:0/3962401776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc610160880 con 0x7fc6100d4d60 2024-09-12T01:34:03.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.181+0000 7fc6166d0640 1 -- 172.21.15.16:0/3962401776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6100b7600 con 0x7fc6100d4d60 2024-09-12T01:34:03.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.182+0000 7fc6166d0640 1 -- 172.21.15.16:0/3962401776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6100cef60 con 0x7fc6100d4d60 2024-09-12T01:34:03.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.183+0000 7fc60d7fa640 1 -- 172.21.15.16:0/3962401776 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc60001b020 con 0x7fc6100d4d60 2024-09-12T01:34:03.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.183+0000 7fc60d7fa640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc5f006be30 0x7fc5f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.183+0000 7fc60d7fa640 1 -- 172.21.15.16:0/3962401776 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fc60008aff0 con 0x7fc6100d4d60 2024-09-12T01:34:03.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.183+0000 7fc607fff640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc5f006be30 0x7fc5f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.184+0000 7fc607fff640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc5f006be30 0x7fc5f006e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8034000 tx=0x7fc5f802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.188+0000 7fc60d7fa640 1 -- 172.21.15.16:0/3962401776 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc600022030 con 0x7fc6100d4d60 2024-09-12T01:34:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.431+0000 7fc6166d0640 1 -- 172.21.15.16:0/3962401776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fc6100b4090 con 0x7fc6100d4d60 2024-09-12T01:34:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.433+0000 7fc60d7fa640 1 -- 172.21.15.16:0/3962401776 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fc600009e20 con 0x7fc6100d4d60 2024-09-12T01:34:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.435+0000 7fc6166d0640 1 -- 172.21.15.16:0/3962401776 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc5f006be30 msgr2=0x7fc5f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.435+0000 7fc6166d0640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc5f006be30 0x7fc5f006e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8034000 tx=0x7fc5f802e000 comp rx=0 tx=0).stop 2024-09-12T01:34:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.435+0000 7fc6166d0640 1 -- 172.21.15.16:0/3962401776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6100d4d60 msgr2=0x7fc610150760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.435+0000 7fc6166d0640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6100d4d60 0x7fc610150760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc60000d830 tx=0x7fc60000dcf0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.435+0000 7fc6166d0640 1 -- 172.21.15.16:0/3962401776 shutdown_connections 2024-09-12T01:34:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.435+0000 7fc6166d0640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fc5f006be30 0x7fc5f006e2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.435+0000 7fc6166d0640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fc610162020 0x7fc6100b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.435+0000 7fc6166d0640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc61015e660 0x7fc61015fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.435+0000 7fc6166d0640 1 --2- 172.21.15.16:0/3962401776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6100d4d60 0x7fc610150760 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.435+0000 7fc6166d0640 1 -- 172.21.15.16:0/3962401776 >> 172.21.15.16:0/3962401776 conn(0x7fc61008de10 msgr2=0x7fc61006d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.435+0000 7fc6166d0640 1 -- 172.21.15.16:0/3962401776 shutdown_connections 2024-09-12T01:34:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.435+0000 7fc6166d0640 1 -- 172.21.15.16:0/3962401776 wait complete. 2024-09-12T01:34:03.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-12T01:34:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.549+0000 7fe2d4133640 1 Processor -- start 2024-09-12T01:34:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.549+0000 7fe2d4133640 1 -- start start 2024-09-12T01:34:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.550+0000 7fe2d4133640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2cc0d4d60 0x7fe2cc0d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.550+0000 7fe2d4133640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe2cc162020 0x7fe2cc0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.550+0000 7fe2d4133640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2cc162500 con 0x7fe2cc15e660 2024-09-12T01:34:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.550+0000 7fe2d4133640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe2cc0d2020 con 0x7fe2cc0d4d60 2024-09-12T01:34:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.550+0000 7fe2d4133640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fe2cc1627e0 con 0x7fe2cc162020 2024-09-12T01:34:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.550+0000 7fe2d16ce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2cc0d4d60 0x7fe2cc0d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.550+0000 7fe2d16ce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2cc0d4d60 0x7fe2cc0d24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52774/0 (socket says 172.21.15.16:52774) 2024-09-12T01:34:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.550+0000 7fe2d16ce640 1 -- 172.21.15.16:0/4103141151 learned_addr learned my addr 172.21.15.16:0/4103141151 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.550+0000 7fe2d16ce640 1 -- 172.21.15.16:0/4103141151 >> v2:172.21.15.33:3300/0 conn(0x7fe2cc162020 msgr2=0x7fe2cc0d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.550+0000 7fe2d16ce640 1 --2- 172.21.15.16:0/4103141151 >> v2:172.21.15.33:3300/0 conn(0x7fe2cc162020 0x7fe2cc0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.551+0000 7fe2d16ce640 1 -- 172.21.15.16:0/4103141151 >> v1:172.21.15.33:6789/0 conn(0x7fe2cc15e660 legacy=0x7fe2cc0d4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.551+0000 7fe2d16ce640 1 -- 172.21.15.16:0/4103141151 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc162ac0 con 0x7fe2cc0d4d60 2024-09-12T01:34:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.551+0000 7fe2d16ce640 1 --2- 172.21.15.16:0/4103141151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc0d4d60 0x7fe2cc0d24b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0d21a0 tx=0x7fe2bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3f414cd4b6658e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.551+0000 7fe2d0ecd640 1 -- 172.21.15.16:0/4103141151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc037070 con 0x7fe2cc0d4d60 2024-09-12T01:34:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.552+0000 7fe2d0ecd640 1 -- 172.21.15.16:0/4103141151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2bc032440 con 0x7fe2cc0d4d60 2024-09-12T01:34:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.552+0000 7fe2d0ecd640 1 -- 172.21.15.16:0/4103141151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc032810 con 0x7fe2cc0d4d60 2024-09-12T01:34:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.552+0000 7fe2d4133640 1 -- 172.21.15.16:0/4103141151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc0d4d60 msgr2=0x7fe2cc0d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.552+0000 7fe2d4133640 1 --2- 172.21.15.16:0/4103141151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc0d4d60 0x7fe2cc0d24b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0d21a0 tx=0x7fe2bc02a180 comp rx=0 tx=0).stop 2024-09-12T01:34:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.552+0000 7fe2d4133640 1 -- 172.21.15.16:0/4103141151 shutdown_connections 2024-09-12T01:34:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.552+0000 7fe2d4133640 1 --2- 172.21.15.16:0/4103141151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc0d4d60 0x7fe2cc0d24b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.552+0000 7fe2d4133640 1 --2- 172.21.15.16:0/4103141151 >> v2:172.21.15.33:3300/0 conn(0x7fe2cc162020 0x7fe2cc0d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.552+0000 7fe2d4133640 1 -- 172.21.15.16:0/4103141151 >> 172.21.15.16:0/4103141151 conn(0x7fe2cc08de10 msgr2=0x7fe2cc09d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.552+0000 7fe2d4133640 1 -- 172.21.15.16:0/4103141151 shutdown_connections 2024-09-12T01:34:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.552+0000 7fe2d4133640 1 -- 172.21.15.16:0/4103141151 wait complete. 2024-09-12T01:34:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.553+0000 7fe2d4133640 1 Processor -- start 2024-09-12T01:34:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.553+0000 7fe2d4133640 1 -- start start 2024-09-12T01:34:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.554+0000 7fe2d4133640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0d4d60 0x7fe2cc161270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.554+0000 7fe2d4133640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc15e660 0x7fe2cc1f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.554+0000 7fe2d4133640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe2cc162020 0x7fe2cc0ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.554+0000 7fe2d4133640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc0d4990 con 0x7fe2cc15e660 2024-09-12T01:34:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.554+0000 7fe2d4133640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc0d4810 con 0x7fe2cc0d4d60 2024-09-12T01:34:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.554+0000 7fe2d4133640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fe2cc0d4b10 con 0x7fe2cc162020 2024-09-12T01:34:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.554+0000 7fe2d16ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc15e660 0x7fe2cc1f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.554+0000 7fe2d16ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc15e660 0x7fe2cc1f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52788/0 (socket says 172.21.15.16:52788) 2024-09-12T01:34:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.554+0000 7fe2d16ce640 1 -- 172.21.15.16:0/2841914004 learned_addr learned my addr 172.21.15.16:0/2841914004 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.554+0000 7fe2d16ce640 1 -- 172.21.15.16:0/2841914004 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe2cc162020 msgr2=0x7fe2cc0ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.554+0000 7fe2d16ce640 1 --2- 172.21.15.16:0/2841914004 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe2cc162020 0x7fe2cc0ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.555+0000 7fe2d16ce640 1 -- 172.21.15.16:0/2841914004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0d4d60 msgr2=0x7fe2cc161270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.555+0000 7fe2d16ce640 1 --2- 172.21.15.16:0/2841914004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0d4d60 0x7fe2cc161270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.555+0000 7fe2d16ce640 1 -- 172.21.15.16:0/2841914004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc15fe20 con 0x7fe2cc15e660 2024-09-12T01:34:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.555+0000 7fe2d16ce640 1 --2- 172.21.15.16:0/2841914004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc15e660 0x7fe2cc1f4f30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc000c00 tx=0x7fe2bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.555+0000 7fe2c27fc640 1 -- 172.21.15.16:0/2841914004 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc041070 con 0x7fe2cc15e660 2024-09-12T01:34:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.555+0000 7fe2c27fc640 1 -- 172.21.15.16:0/2841914004 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2bc03c870 con 0x7fe2cc15e660 2024-09-12T01:34:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.555+0000 7fe2d4133640 1 -- 172.21.15.16:0/2841914004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2cc1f5470 con 0x7fe2cc15e660 2024-09-12T01:34:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.555+0000 7fe2c27fc640 1 -- 172.21.15.16:0/2841914004 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc037040 con 0x7fe2cc15e660 2024-09-12T01:34:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.555+0000 7fe2d4133640 1 -- 172.21.15.16:0/2841914004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2cc15f060 con 0x7fe2cc15e660 2024-09-12T01:34:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.556+0000 7fe2d4133640 1 -- 172.21.15.16:0/2841914004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2cc0d0f50 con 0x7fe2cc15e660 2024-09-12T01:34:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.556+0000 7fe2c27fc640 1 -- 172.21.15.16:0/2841914004 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2bc044070 con 0x7fe2cc15e660 2024-09-12T01:34:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.557+0000 7fe2c27fc640 1 --2- 172.21.15.16:0/2841914004 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe2b006be60 0x7fe2b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.557+0000 7fe2c27fc640 1 -- 172.21.15.16:0/2841914004 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fe2bc0ab910 con 0x7fe2cc15e660 2024-09-12T01:34:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.557+0000 7fe2d1ecf640 1 --2- 172.21.15.16:0/2841914004 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe2b006be60 0x7fe2b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.557+0000 7fe2d1ecf640 1 --2- 172.21.15.16:0/2841914004 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe2b006be60 0x7fe2b006e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8001560 tx=0x7fe2c8007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.561+0000 7fe2c27fc640 1 -- 172.21.15.16:0/2841914004 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe2cc1f5470 con 0x7fe2cc15e660 2024-09-12T01:34:03.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.802+0000 7fe2d4133640 1 -- 172.21.15.16:0/2841914004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fe2cc0b4090 con 0x7fe2cc15e660 2024-09-12T01:34:03.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.804+0000 7fe2c27fc640 1 -- 172.21.15.16:0/2841914004 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7fe2bc07a630 con 0x7fe2cc15e660 2024-09-12T01:34:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.806+0000 7fe2d4133640 1 -- 172.21.15.16:0/2841914004 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe2b006be60 msgr2=0x7fe2b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.806+0000 7fe2d4133640 1 --2- 172.21.15.16:0/2841914004 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe2b006be60 0x7fe2b006e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8001560 tx=0x7fe2c8007990 comp rx=0 tx=0).stop 2024-09-12T01:34:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.806+0000 7fe2d4133640 1 -- 172.21.15.16:0/2841914004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc15e660 msgr2=0x7fe2cc1f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.806+0000 7fe2d4133640 1 --2- 172.21.15.16:0/2841914004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc15e660 0x7fe2cc1f4f30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc000c00 tx=0x7fe2bc0046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.806+0000 7fe2d4133640 1 -- 172.21.15.16:0/2841914004 shutdown_connections 2024-09-12T01:34:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.806+0000 7fe2d4133640 1 --2- 172.21.15.16:0/2841914004 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe2b006be60 0x7fe2b006e320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.806+0000 7fe2d4133640 1 --2- 172.21.15.16:0/2841914004 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe2cc162020 0x7fe2cc0ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.806+0000 7fe2d4133640 1 --2- 172.21.15.16:0/2841914004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc15e660 0x7fe2cc1f4f30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.806+0000 7fe2d4133640 1 --2- 172.21.15.16:0/2841914004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0d4d60 0x7fe2cc161270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.806+0000 7fe2d4133640 1 -- 172.21.15.16:0/2841914004 >> 172.21.15.16:0/2841914004 conn(0x7fe2cc08de10 msgr2=0x7fe2cc06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.806+0000 7fe2d4133640 1 -- 172.21.15.16:0/2841914004 shutdown_connections 2024-09-12T01:34:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.806+0000 7fe2d4133640 1 -- 172.21.15.16:0/2841914004 wait complete. 2024-09-12T01:34:03.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-12T01:34:03.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.919+0000 7f5c62447640 1 Processor -- start 2024-09-12T01:34:03.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.920+0000 7f5c62447640 1 -- start start 2024-09-12T01:34:03.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.920+0000 7f5c62447640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5c5c15c130 0x7f5c5c0d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.920+0000 7f5c62447640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c5c15ffe0 0x7f5c5c0d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.920+0000 7f5c62447640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c5c0d1fb0 con 0x7f5c5c0d0750 2024-09-12T01:34:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.920+0000 7f5c62447640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5c5c1604a0 con 0x7f5c5c15c130 2024-09-12T01:34:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.920+0000 7f5c62447640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5c160780 con 0x7f5c5c15ffe0 2024-09-12T01:34:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.921+0000 7f5c609e4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5c5c0d0750 0x7f5c5c0cecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48320/0 (socket says 172.21.15.16:48320) 2024-09-12T01:34:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.921+0000 7f5c609e4640 1 -- 172.21.15.16:0/3771785447 learned_addr learned my addr 172.21.15.16:0/3771785447 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.921+0000 7f5c5bfff640 1 --2- 172.21.15.16:0/3771785447 >> v2:172.21.15.16:3300/0 conn(0x7f5c5c15c130 0x7f5c5c0d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.921+0000 7f5c53fff640 1 --2- 172.21.15.16:0/3771785447 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c5c15ffe0 0x7f5c5c0d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.921+0000 7f5c5bfff640 1 -- 172.21.15.16:0/3771785447 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c5c15ffe0 msgr2=0x7f5c5c0d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.921+0000 7f5c5bfff640 1 --2- 172.21.15.16:0/3771785447 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c5c15ffe0 0x7f5c5c0d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.921+0000 7f5c5bfff640 1 -- 172.21.15.16:0/3771785447 >> v1:172.21.15.16:6789/0 conn(0x7f5c5c0d0750 legacy=0x7f5c5c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.921+0000 7f5c5bfff640 1 -- 172.21.15.16:0/3771785447 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c160a60 con 0x7f5c5c15c130 2024-09-12T01:34:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.921+0000 7f5c5b7fe640 1 -- 172.21.15.16:0/3771785447 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1763665179 0 0) 0x7f5c5c0d1fb0 con 0x7f5c5c0d0750 2024-09-12T01:34:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.922+0000 7f5c5bfff640 1 --2- 172.21.15.16:0/3771785447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5c15c130 0x7f5c5c0d1a70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0097a0 tx=0x7f5c4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660d06b0aa31f171 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.922+0000 7f5c5b7fe640 1 -- 172.21.15.16:0/3771785447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c037070 con 0x7f5c5c15c130 2024-09-12T01:34:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.922+0000 7f5c5b7fe640 1 -- 172.21.15.16:0/3771785447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c032440 con 0x7f5c5c15c130 2024-09-12T01:34:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.922+0000 7f5c5b7fe640 1 -- 172.21.15.16:0/3771785447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c032810 con 0x7f5c5c15c130 2024-09-12T01:34:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.922+0000 7f5c62447640 1 -- 172.21.15.16:0/3771785447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5c15c130 msgr2=0x7f5c5c0d1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.922+0000 7f5c62447640 1 --2- 172.21.15.16:0/3771785447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5c15c130 0x7f5c5c0d1a70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0097a0 tx=0x7f5c4c02a180 comp rx=0 tx=0).stop 2024-09-12T01:34:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.923+0000 7f5c62447640 1 -- 172.21.15.16:0/3771785447 shutdown_connections 2024-09-12T01:34:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.923+0000 7f5c62447640 1 --2- 172.21.15.16:0/3771785447 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c5c15ffe0 0x7f5c5c0d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.923+0000 7f5c62447640 1 --2- 172.21.15.16:0/3771785447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5c15c130 0x7f5c5c0d1a70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.923+0000 7f5c62447640 1 -- 172.21.15.16:0/3771785447 >> 172.21.15.16:0/3771785447 conn(0x7f5c5c06baf0 msgr2=0x7f5c5c06bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.923+0000 7f5c62447640 1 -- 172.21.15.16:0/3771785447 shutdown_connections 2024-09-12T01:34:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.923+0000 7f5c62447640 1 -- 172.21.15.16:0/3771785447 wait complete. 2024-09-12T01:34:03.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.924+0000 7f5c62447640 1 Processor -- start 2024-09-12T01:34:03.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.924+0000 7f5c62447640 1 -- start start 2024-09-12T01:34:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c62447640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c5c0d0750 0x7f5c5c1f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c62447640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5c15c130 0x7f5c5c0c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c62447640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c5c15ffe0 0x7f5c5c15a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c62447640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c0cf3d0 con 0x7f5c5c15c130 2024-09-12T01:34:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c62447640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c0d2200 con 0x7f5c5c15ffe0 2024-09-12T01:34:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c62447640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5c0cf550 con 0x7f5c5c0d0750 2024-09-12T01:34:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5c15c130 0x7f5c5c0c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5c15c130 0x7f5c5c0c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52804/0 (socket says 172.21.15.16:52804) 2024-09-12T01:34:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c53fff640 1 -- 172.21.15.16:0/1242415634 learned_addr learned my addr 172.21.15.16:0/1242415634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c5bfff640 1 --2- 172.21.15.16:0/1242415634 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c5c0d0750 0x7f5c5c1f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c53fff640 1 -- 172.21.15.16:0/1242415634 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c5c0d0750 msgr2=0x7f5c5c1f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c53fff640 1 --2- 172.21.15.16:0/1242415634 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c5c0d0750 0x7f5c5c1f76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c53fff640 1 -- 172.21.15.16:0/1242415634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c5c15ffe0 msgr2=0x7f5c5c15a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c53fff640 1 --2- 172.21.15.16:0/1242415634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c5c15ffe0 0x7f5c5c15a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c53fff640 1 -- 172.21.15.16:0/1242415634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c1f85e0 con 0x7f5c5c15c130 2024-09-12T01:34:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c53fff640 1 --2- 172.21.15.16:0/1242415634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5c15c130 0x7f5c5c0c6e30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400e9b0 tx=0x7f5c4400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c597fa640 1 -- 172.21.15.16:0/1242415634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44017070 con 0x7f5c5c15c130 2024-09-12T01:34:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c597fa640 1 -- 172.21.15.16:0/1242415634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4400c9a0 con 0x7f5c5c15c130 2024-09-12T01:34:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c62447640 1 -- 172.21.15.16:0/1242415634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c1f7d40 con 0x7f5c5c15c130 2024-09-12T01:34:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c597fa640 1 -- 172.21.15.16:0/1242415634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4400cd90 con 0x7f5c5c15c130 2024-09-12T01:34:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.925+0000 7f5c62447640 1 -- 172.21.15.16:0/1242415634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c1f8300 con 0x7f5c5c15c130 2024-09-12T01:34:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.926+0000 7f5c62447640 1 -- 172.21.15.16:0/1242415634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c5c068880 con 0x7f5c5c15c130 2024-09-12T01:34:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.927+0000 7f5c597fa640 1 -- 172.21.15.16:0/1242415634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c4401a070 con 0x7f5c5c15c130 2024-09-12T01:34:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.927+0000 7f5c597fa640 1 --2- 172.21.15.16:0/1242415634 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5c3c06be30 0x7f5c3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.927+0000 7f5c597fa640 1 -- 172.21.15.16:0/1242415634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f5c4408ac00 con 0x7f5c5c15c130 2024-09-12T01:34:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.927+0000 7f5c5bfff640 1 --2- 172.21.15.16:0/1242415634 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5c3c06be30 0x7f5c3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.928+0000 7f5c5bfff640 1 --2- 172.21.15.16:0/1242415634 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5c3c06be30 0x7f5c3c06e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c034000 tx=0x7f5c4c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:03.931+0000 7f5c597fa640 1 -- 172.21.15.16:0/1242415634 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5c44091080 con 0x7f5c5c15c130 2024-09-12T01:34:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.177+0000 7f5c62447640 1 -- 172.21.15.16:0/1242415634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f5c5c0b1cf0 con 0x7f5c5c15c130 2024-09-12T01:34:04.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.179+0000 7f5c597fa640 1 -- 172.21.15.16:0/1242415634 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f5c44059800 con 0x7f5c5c15c130 2024-09-12T01:34:04.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.181+0000 7f5c62447640 1 -- 172.21.15.16:0/1242415634 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5c3c06be30 msgr2=0x7f5c3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:04.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.181+0000 7f5c62447640 1 --2- 172.21.15.16:0/1242415634 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5c3c06be30 0x7f5c3c06e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c034000 tx=0x7f5c4c02e000 comp rx=0 tx=0).stop 2024-09-12T01:34:04.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.181+0000 7f5c62447640 1 -- 172.21.15.16:0/1242415634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5c15c130 msgr2=0x7f5c5c0c6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:04.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.181+0000 7f5c62447640 1 --2- 172.21.15.16:0/1242415634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5c15c130 0x7f5c5c0c6e30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400e9b0 tx=0x7f5c4400ee70 comp rx=0 tx=0).stop 2024-09-12T01:34:04.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.181+0000 7f5c62447640 1 -- 172.21.15.16:0/1242415634 shutdown_connections 2024-09-12T01:34:04.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.181+0000 7f5c62447640 1 --2- 172.21.15.16:0/1242415634 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f5c3c06be30 0x7f5c3c06e2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.181+0000 7f5c62447640 1 --2- 172.21.15.16:0/1242415634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c5c15ffe0 0x7f5c5c15a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.181+0000 7f5c62447640 1 --2- 172.21.15.16:0/1242415634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5c15c130 0x7f5c5c0c6e30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.181+0000 7f5c62447640 1 --2- 172.21.15.16:0/1242415634 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c5c0d0750 0x7f5c5c1f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.181+0000 7f5c62447640 1 -- 172.21.15.16:0/1242415634 >> 172.21.15.16:0/1242415634 conn(0x7f5c5c06baf0 msgr2=0x7f5c5c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:04.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.181+0000 7f5c62447640 1 -- 172.21.15.16:0/1242415634 shutdown_connections 2024-09-12T01:34:04.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.181+0000 7f5c62447640 1 -- 172.21.15.16:0/1242415634 wait complete. 2024-09-12T01:34:04.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T01:34:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.294+0000 7f6a3f31f640 1 Processor -- start 2024-09-12T01:34:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.294+0000 7f6a3f31f640 1 -- start start 2024-09-12T01:34:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.295+0000 7f6a3f31f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6a3815e5a0 0x7f6a380d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.295+0000 7f6a3f31f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a380d2d70 0x7f6a380d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.295+0000 7f6a3f31f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a380d16b0 con 0x7f6a380d1200 2024-09-12T01:34:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.295+0000 7f6a3f31f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6a3815e980 con 0x7f6a380d2d70 2024-09-12T01:34:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.295+0000 7f6a3f31f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f6a380d3ee0 con 0x7f6a3815e5a0 2024-09-12T01:34:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.295+0000 7f6a3c8ba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a380d2d70 0x7f6a380d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.295+0000 7f6a3d8bc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6a380d1200 0x7f6a380b5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48326/0 (socket says 172.21.15.16:48326) 2024-09-12T01:34:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.295+0000 7f6a3d8bc640 1 -- 172.21.15.16:0/2522343945 learned_addr learned my addr 172.21.15.16:0/2522343945 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.295+0000 7f6a3d0bb640 1 --2- 172.21.15.16:0/2522343945 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6a3815e5a0 0x7f6a380d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.296+0000 7f6a3c8ba640 1 -- 172.21.15.16:0/2522343945 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6a3815e5a0 msgr2=0x7f6a380d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.296+0000 7f6a3c8ba640 1 --2- 172.21.15.16:0/2522343945 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6a3815e5a0 0x7f6a380d2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.296+0000 7f6a3c8ba640 1 -- 172.21.15.16:0/2522343945 >> v1:172.21.15.16:6789/0 conn(0x7f6a380d1200 legacy=0x7f6a380b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.296+0000 7f6a2ffff640 1 -- 172.21.15.16:0/2522343945 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4001538047 0 0) 0x7f6a30001490 con 0x7f6a380d1200 2024-09-12T01:34:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.296+0000 7f6a3c8ba640 1 -- 172.21.15.16:0/2522343945 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a380d32d0 con 0x7f6a380d2d70 2024-09-12T01:34:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.296+0000 7f6a3c8ba640 1 --2- 172.21.15.16:0/2522343945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a380d2d70 0x7f6a380d0cc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6a280097a0 tx=0x7f6a2802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb642641ea8aa914 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.296+0000 7f6a2ffff640 1 -- 172.21.15.16:0/2522343945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28037070 con 0x7f6a380d2d70 2024-09-12T01:34:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.296+0000 7f6a2ffff640 1 -- 172.21.15.16:0/2522343945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a28032440 con 0x7f6a380d2d70 2024-09-12T01:34:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.297+0000 7f6a2ffff640 1 -- 172.21.15.16:0/2522343945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28032810 con 0x7f6a380d2d70 2024-09-12T01:34:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.297+0000 7f6a3f31f640 1 -- 172.21.15.16:0/2522343945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a380d2d70 msgr2=0x7f6a380d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.297+0000 7f6a3f31f640 1 --2- 172.21.15.16:0/2522343945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a380d2d70 0x7f6a380d0cc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6a280097a0 tx=0x7f6a2802a180 comp rx=0 tx=0).stop 2024-09-12T01:34:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.297+0000 7f6a3f31f640 1 -- 172.21.15.16:0/2522343945 shutdown_connections 2024-09-12T01:34:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.297+0000 7f6a3f31f640 1 --2- 172.21.15.16:0/2522343945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a380d2d70 0x7f6a380d0cc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.297+0000 7f6a3f31f640 1 --2- 172.21.15.16:0/2522343945 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6a3815e5a0 0x7f6a380d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.297+0000 7f6a3f31f640 1 -- 172.21.15.16:0/2522343945 >> 172.21.15.16:0/2522343945 conn(0x7f6a3806d1c0 msgr2=0x7f6a38073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.297+0000 7f6a3f31f640 1 -- 172.21.15.16:0/2522343945 shutdown_connections 2024-09-12T01:34:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.297+0000 7f6a3f31f640 1 -- 172.21.15.16:0/2522343945 wait complete. 2024-09-12T01:34:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.298+0000 7f6a3f31f640 1 Processor -- start 2024-09-12T01:34:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.298+0000 7f6a3f31f640 1 -- start start 2024-09-12T01:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.299+0000 7f6a3f31f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6a380d1200 0x7f6a380cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.299+0000 7f6a3f31f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a380d2d70 0x7f6a380cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.299+0000 7f6a3f31f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a3815e5a0 0x7f6a381f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.299+0000 7f6a3f31f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a380d3b70 con 0x7f6a3815e5a0 2024-09-12T01:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.299+0000 7f6a3f31f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a380d3890 con 0x7f6a380d2d70 2024-09-12T01:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.299+0000 7f6a3f31f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f6a380d22f0 con 0x7f6a380d1200 2024-09-12T01:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.299+0000 7f6a3d8bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a3815e5a0 0x7f6a381f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.299+0000 7f6a3d8bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a3815e5a0 0x7f6a381f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52824/0 (socket says 172.21.15.16:52824) 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.299+0000 7f6a3d8bc640 1 -- 172.21.15.16:0/1486838122 learned_addr learned my addr 172.21.15.16:0/1486838122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.299+0000 7f6a3d0bb640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6a380d1200 0x7f6a380cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.299+0000 7f6a3c8ba640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a380d2d70 0x7f6a380cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.300+0000 7f6a3d8bc640 1 -- 172.21.15.16:0/1486838122 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6a380d1200 msgr2=0x7f6a380cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.300+0000 7f6a3d8bc640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6a380d1200 0x7f6a380cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.300+0000 7f6a3d8bc640 1 -- 172.21.15.16:0/1486838122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a380d2d70 msgr2=0x7f6a380cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.300+0000 7f6a3d8bc640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a380d2d70 0x7f6a380cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.300+0000 7f6a3d8bc640 1 -- 172.21.15.16:0/1486838122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a380ce400 con 0x7f6a3815e5a0 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.300+0000 7f6a3d8bc640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a3815e5a0 0x7f6a381f3b30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000c7f0 tx=0x7f6a3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.300+0000 7f6a2dffb640 1 -- 172.21.15.16:0/1486838122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30018070 con 0x7f6a3815e5a0 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.300+0000 7f6a2dffb640 1 -- 172.21.15.16:0/1486838122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3000a5e0 con 0x7f6a3815e5a0 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.300+0000 7f6a3f31f640 1 -- 172.21.15.16:0/1486838122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a380ce6e0 con 0x7f6a3815e5a0 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.300+0000 7f6a2dffb640 1 -- 172.21.15.16:0/1486838122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3000aa10 con 0x7f6a3815e5a0 2024-09-12T01:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.300+0000 7f6a3f31f640 1 -- 172.21.15.16:0/1486838122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a380c8af0 con 0x7f6a3815e5a0 2024-09-12T01:34:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.301+0000 7f6a3f31f640 1 -- 172.21.15.16:0/1486838122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a380d0cc0 con 0x7f6a3815e5a0 2024-09-12T01:34:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.302+0000 7f6a2dffb640 1 -- 172.21.15.16:0/1486838122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a3001b070 con 0x7f6a3815e5a0 2024-09-12T01:34:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.302+0000 7f6a2dffb640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6a1c06be30 0x7f6a1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.302+0000 7f6a2dffb640 1 -- 172.21.15.16:0/1486838122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f6a3008a1f0 con 0x7f6a3815e5a0 2024-09-12T01:34:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.302+0000 7f6a3d0bb640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6a1c06be30 0x7f6a1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.303+0000 7f6a3d0bb640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6a1c06be30 0x7f6a1c06e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400aa20 tx=0x7f6a34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:04.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.306+0000 7f6a2dffb640 1 -- 172.21.15.16:0/1486838122 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6a380d16b0 con 0x7f6a3815e5a0 2024-09-12T01:34:04.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.481+0000 7f6a3f31f640 1 -- 172.21.15.16:0/1486838122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6a380b8af0 con 0x7f6a3815e5a0 2024-09-12T01:34:04.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.482+0000 7f6a2dffb640 1 -- 172.21.15.16:0/1486838122 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) ==== 84+0+65 (secure 0 0 0) 0x7f6a30058710 con 0x7f6a3815e5a0 2024-09-12T01:34:04.483 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:04.483 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-12T01:34:04.483 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:34:04.483 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-12T01:34:04.483 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-12T01:34:04.483 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:04.483 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-12T01:34:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.484+0000 7f6a3f31f640 1 -- 172.21.15.16:0/1486838122 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6a1c06be30 msgr2=0x7f6a1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.484+0000 7f6a3f31f640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6a1c06be30 0x7f6a1c06e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400aa20 tx=0x7f6a34008040 comp rx=0 tx=0).stop 2024-09-12T01:34:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.484+0000 7f6a3f31f640 1 -- 172.21.15.16:0/1486838122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a3815e5a0 msgr2=0x7f6a381f3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.484+0000 7f6a3f31f640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a3815e5a0 0x7f6a381f3b30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000c7f0 tx=0x7f6a3000ccb0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.484+0000 7f6a3f31f640 1 -- 172.21.15.16:0/1486838122 shutdown_connections 2024-09-12T01:34:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.484+0000 7f6a3f31f640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f6a1c06be30 0x7f6a1c06e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.484+0000 7f6a3f31f640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a3815e5a0 0x7f6a381f3b30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.484+0000 7f6a3f31f640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a380d2d70 0x7f6a380cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.484+0000 7f6a3f31f640 1 --2- 172.21.15.16:0/1486838122 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f6a380d1200 0x7f6a380cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.484+0000 7f6a3f31f640 1 -- 172.21.15.16:0/1486838122 >> 172.21.15.16:0/1486838122 conn(0x7f6a3806d1c0 msgr2=0x7f6a38161300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.484+0000 7f6a3f31f640 1 -- 172.21.15.16:0/1486838122 shutdown_connections 2024-09-12T01:34:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.484+0000 7f6a3f31f640 1 -- 172.21.15.16:0/1486838122 wait complete. 2024-09-12T01:34:04.496 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T01:34:04.496 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T01:34:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.602+0000 7f8681ae6640 1 Processor -- start 2024-09-12T01:34:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.602+0000 7f8681ae6640 1 -- start start 2024-09-12T01:34:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.603+0000 7f8681ae6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f867c0d13a0 0x7f867c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.603+0000 7f8681ae6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867c0d1fb0 con 0x7f867c0d3050 2024-09-12T01:34:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.603+0000 7f8681ae6640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867c0d2290 con 0x7f867c15e4d0 2024-09-12T01:34:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.603+0000 7f8681ae6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f867c0d2570 con 0x7f867c0d13a0 2024-09-12T01:34:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.603+0000 7f867bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f867c0d13a0 0x7f867c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.603+0000 7f867affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f867c0d3050 0x7f867c0d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48338/0 (socket says 172.21.15.16:48338) 2024-09-12T01:34:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.603+0000 7f867affd640 1 -- 172.21.15.16:0/852343443 learned_addr learned my addr 172.21.15.16:0/852343443 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.603+0000 7f867bfff640 1 -- 172.21.15.16:0/852343443 >> v1:172.21.15.33:6789/0 conn(0x7f867c15e4d0 legacy=0x7f867c0cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.603+0000 7f867bfff640 1 -- 172.21.15.16:0/852343443 >> v1:172.21.15.16:6789/0 conn(0x7f867c0d3050 legacy=0x7f867c0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.603+0000 7f867bfff640 1 -- 172.21.15.16:0/852343443 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c0d2850 con 0x7f867c0d13a0 2024-09-12T01:34:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.604+0000 7f867bfff640 1 --2- 172.21.15.16:0/852343443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c0d13a0 0x7f867c0b3d00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8674009d90 tx=0x7f867402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153dd37d4718b238 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.604+0000 7f867a7fc640 1 -- 172.21.15.16:0/852343443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674031600 con 0x7f867c0d13a0 2024-09-12T01:34:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.604+0000 7f867a7fc640 1 -- 172.21.15.16:0/852343443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86740317a0 con 0x7f867c0d13a0 2024-09-12T01:34:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.604+0000 7f867a7fc640 1 -- 172.21.15.16:0/852343443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674031b70 con 0x7f867c0d13a0 2024-09-12T01:34:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.605+0000 7f8681ae6640 1 -- 172.21.15.16:0/852343443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c0d13a0 msgr2=0x7f867c0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.605+0000 7f8681ae6640 1 --2- 172.21.15.16:0/852343443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c0d13a0 0x7f867c0b3d00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8674009d90 tx=0x7f867402a610 comp rx=0 tx=0).stop 2024-09-12T01:34:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.605+0000 7f8681ae6640 1 -- 172.21.15.16:0/852343443 shutdown_connections 2024-09-12T01:34:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.605+0000 7f8681ae6640 1 --2- 172.21.15.16:0/852343443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c0d13a0 0x7f867c0b3d00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.605+0000 7f8681ae6640 1 -- 172.21.15.16:0/852343443 >> 172.21.15.16:0/852343443 conn(0x7f867c09d650 msgr2=0x7f867c0d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:04.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.605+0000 7f8681ae6640 1 -- 172.21.15.16:0/852343443 shutdown_connections 2024-09-12T01:34:04.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.605+0000 7f8681ae6640 1 -- 172.21.15.16:0/852343443 wait complete. 2024-09-12T01:34:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.606+0000 7f8681ae6640 1 Processor -- start 2024-09-12T01:34:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.606+0000 7f8681ae6640 1 -- start start 2024-09-12T01:34:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f8681ae6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c0d13a0 0x7f867c14c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f8681ae6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f867c0d3050 0x7f867c0c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f8681ae6640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f867c15e4d0 0x7f867c0c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f8681ae6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f867c0d01a0 con 0x7f867c0d13a0 2024-09-12T01:34:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f8681ae6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f867c0d4d80 con 0x7f867c0d3050 2024-09-12T01:34:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f8681ae6640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f867c0d0320 con 0x7f867c15e4d0 2024-09-12T01:34:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f867b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c0d13a0 0x7f867c14c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f867b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c0d13a0 0x7f867c14c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52844/0 (socket says 172.21.15.16:52844) 2024-09-12T01:34:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f867b7fe640 1 -- 172.21.15.16:0/2819185872 learned_addr learned my addr 172.21.15.16:0/2819185872 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f867affd640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f867c0d3050 0x7f867c0c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f867bfff640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f867c15e4d0 0x7f867c0c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f867b7fe640 1 -- 172.21.15.16:0/2819185872 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f867c15e4d0 msgr2=0x7f867c0c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f867b7fe640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f867c15e4d0 0x7f867c0c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f867b7fe640 1 -- 172.21.15.16:0/2819185872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f867c0d3050 msgr2=0x7f867c0c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f867b7fe640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f867c0d3050 0x7f867c0c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.607+0000 7f867b7fe640 1 -- 172.21.15.16:0/2819185872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c0ca830 con 0x7f867c0d13a0 2024-09-12T01:34:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.608+0000 7f867b7fe640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c0d13a0 0x7f867c14c870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f866c00b3f0 tx=0x7f866c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.608+0000 7f8663fff640 1 -- 172.21.15.16:0/2819185872 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c015020 con 0x7f867c0d13a0 2024-09-12T01:34:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.608+0000 7f8663fff640 1 -- 172.21.15.16:0/2819185872 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866c011830 con 0x7f867c0d13a0 2024-09-12T01:34:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.608+0000 7f8663fff640 1 -- 172.21.15.16:0/2819185872 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c011c20 con 0x7f867c0d13a0 2024-09-12T01:34:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.608+0000 7f8681ae6640 1 -- 172.21.15.16:0/2819185872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867c0cab10 con 0x7f867c0d13a0 2024-09-12T01:34:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.608+0000 7f8681ae6640 1 -- 172.21.15.16:0/2819185872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c0cb0d0 con 0x7f867c0d13a0 2024-09-12T01:34:04.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.609+0000 7f8681ae6640 1 -- 172.21.15.16:0/2819185872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867c069110 con 0x7f867c0d13a0 2024-09-12T01:34:04.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.609+0000 7f8663fff640 1 -- 172.21.15.16:0/2819185872 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f866c018020 con 0x7f867c0d13a0 2024-09-12T01:34:04.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.610+0000 7f8663fff640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f866806be30 0x7f866806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:04.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.610+0000 7f8663fff640 1 -- 172.21.15.16:0/2819185872 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f866c08a810 con 0x7f867c0d13a0 2024-09-12T01:34:04.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.610+0000 7f867affd640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f866806be30 0x7f866806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:04.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.610+0000 7f867affd640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f866806be30 0x7f866806e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f867c1f7b00 tx=0x7f8670007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:04.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.614+0000 7f8663fff640 1 -- 172.21.15.16:0/2819185872 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f866c01f030 con 0x7f867c0d13a0 2024-09-12T01:34:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:04.791+0000 7f8681ae6640 1 -- 172.21.15.16:0/2819185872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f867c15e8b0 con 0x7f867c0d13a0 2024-09-12T01:34:05.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.543+0000 7f8663fff640 1 -- 172.21.15.16:0/2819185872 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) ==== 138+0+0 (secure 0 0 0) 0x7f866c058d30 con 0x7f867c0d13a0 2024-09-12T01:34:05.544 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-12T01:34:05.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.547+0000 7f8681ae6640 1 -- 172.21.15.16:0/2819185872 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f866806be30 msgr2=0x7f866806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:05.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.547+0000 7f8681ae6640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f866806be30 0x7f866806e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f867c1f7b00 tx=0x7f8670007e80 comp rx=0 tx=0).stop 2024-09-12T01:34:05.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.547+0000 7f8681ae6640 1 -- 172.21.15.16:0/2819185872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c0d13a0 msgr2=0x7f867c14c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:05.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.547+0000 7f8681ae6640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c0d13a0 0x7f867c14c870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f866c00b3f0 tx=0x7f866c00b8b0 comp rx=0 tx=0).stop 2024-09-12T01:34:05.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.547+0000 7f8681ae6640 1 -- 172.21.15.16:0/2819185872 shutdown_connections 2024-09-12T01:34:05.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.547+0000 7f8681ae6640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f866806be30 0x7f866806e2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.547+0000 7f8681ae6640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f867c15e4d0 0x7f867c0c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.547+0000 7f8681ae6640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f867c0d3050 0x7f867c0c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.547+0000 7f8681ae6640 1 --2- 172.21.15.16:0/2819185872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c0d13a0 0x7f867c14c870 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.547+0000 7f8681ae6640 1 -- 172.21.15.16:0/2819185872 >> 172.21.15.16:0/2819185872 conn(0x7f867c09d650 msgr2=0x7f867c0b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.548+0000 7f8681ae6640 1 -- 172.21.15.16:0/2819185872 shutdown_connections 2024-09-12T01:34:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.549+0000 7f8681ae6640 1 -- 172.21.15.16:0/2819185872 wait complete. 2024-09-12T01:34:05.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T01:34:05.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.681+0000 7f58d372a640 1 Processor -- start 2024-09-12T01:34:05.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.681+0000 7f58d372a640 1 -- start start 2024-09-12T01:34:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.681+0000 7f58d372a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58cc0b3a40 0x7f58cc0b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.681+0000 7f58d372a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc0cfb80 0x7f58cc0b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.681+0000 7f58d372a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58cc161ee0 con 0x7f58cc161b00 2024-09-12T01:34:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.681+0000 7f58d372a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f58cc0ceb70 con 0x7f58cc0b3a40 2024-09-12T01:34:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.681+0000 7f58d372a640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc0cf6f0 con 0x7f58cc0cfb80 2024-09-12T01:34:05.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.682+0000 7f58d0cc5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58cc0b3a40 0x7f58cc0b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:05.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.682+0000 7f58d1cc7640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc0cfb80 0x7f58cc0b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:05.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.682+0000 7f58d14c6640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f58cc161b00 0x7f58cc0d3030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:35388/0 (socket says 172.21.15.16:35388) 2024-09-12T01:34:05.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.682+0000 7f58d0cc5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58cc0b3a40 0x7f58cc0b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:45176/0 (socket says 172.21.15.16:45176) 2024-09-12T01:34:05.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.682+0000 7f58d14c6640 1 -- 172.21.15.16:0/986202549 learned_addr learned my addr 172.21.15.16:0/986202549 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:05.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.682+0000 7f58d0cc5640 1 -- 172.21.15.16:0/986202549 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc0cfb80 msgr2=0x7f58cc0b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:05.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.682+0000 7f58d0cc5640 1 --2- 172.21.15.16:0/986202549 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc0cfb80 0x7f58cc0b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:05.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.682+0000 7f58d0cc5640 1 -- 172.21.15.16:0/986202549 >> v1:172.21.15.33:6789/0 conn(0x7f58cc161b00 legacy=0x7f58cc0d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:05.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.682+0000 7f58c3fff640 1 -- 172.21.15.16:0/986202549 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 356719121 0 0) 0x7f58cc161ee0 con 0x7f58cc161b00 2024-09-12T01:34:05.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.682+0000 7f58d0cc5640 1 -- 172.21.15.16:0/986202549 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc0d3740 con 0x7f58cc0b3a40 2024-09-12T01:34:05.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.683+0000 7f58d0cc5640 1 --2- 172.21.15.16:0/986202549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc0b3a40 0x7f58cc0b5fa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0097a0 tx=0x7f58bc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f94d38a19450d50a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:05.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.683+0000 7f58c3fff640 1 -- 172.21.15.16:0/986202549 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc037070 con 0x7f58cc0b3a40 2024-09-12T01:34:05.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.683+0000 7f58c3fff640 1 -- 172.21.15.16:0/986202549 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58bc032440 con 0x7f58cc0b3a40 2024-09-12T01:34:05.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.683+0000 7f58c3fff640 1 -- 172.21.15.16:0/986202549 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc032810 con 0x7f58cc0b3a40 2024-09-12T01:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.684+0000 7f58d372a640 1 -- 172.21.15.16:0/986202549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc0b3a40 msgr2=0x7f58cc0b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.684+0000 7f58d372a640 1 --2- 172.21.15.16:0/986202549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc0b3a40 0x7f58cc0b5fa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0097a0 tx=0x7f58bc02a180 comp rx=0 tx=0).stop 2024-09-12T01:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.684+0000 7f58d372a640 1 -- 172.21.15.16:0/986202549 shutdown_connections 2024-09-12T01:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.684+0000 7f58d372a640 1 --2- 172.21.15.16:0/986202549 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc0cfb80 0x7f58cc0b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.684+0000 7f58d372a640 1 --2- 172.21.15.16:0/986202549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc0b3a40 0x7f58cc0b5fa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.684+0000 7f58d372a640 1 -- 172.21.15.16:0/986202549 >> 172.21.15.16:0/986202549 conn(0x7f58cc08bfc0 msgr2=0x7f58cc09d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.684+0000 7f58d372a640 1 -- 172.21.15.16:0/986202549 shutdown_connections 2024-09-12T01:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.684+0000 7f58d372a640 1 -- 172.21.15.16:0/986202549 wait complete. 2024-09-12T01:34:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.685+0000 7f58d372a640 1 Processor -- start 2024-09-12T01:34:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.685+0000 7f58d372a640 1 -- start start 2024-09-12T01:34:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d372a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0b3a40 0x7f58cc0a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d372a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc0cfb80 0x7f58cc15bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d372a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc161b00 0x7f58cc0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d372a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc0cf380 con 0x7f58cc0b3a40 2024-09-12T01:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d372a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc0cf0a0 con 0x7f58cc0cfb80 2024-09-12T01:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d372a640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc06cc30 con 0x7f58cc161b00 2024-09-12T01:34:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d14c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0b3a40 0x7f58cc0a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d14c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0b3a40 0x7f58cc0a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52848/0 (socket says 172.21.15.16:52848) 2024-09-12T01:34:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d14c6640 1 -- 172.21.15.16:0/3263943397 learned_addr learned my addr 172.21.15.16:0/3263943397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d1cc7640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc161b00 0x7f58cc0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d0cc5640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc0cfb80 0x7f58cc15bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d14c6640 1 -- 172.21.15.16:0/3263943397 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc161b00 msgr2=0x7f58cc0cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d14c6640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc161b00 0x7f58cc0cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d14c6640 1 -- 172.21.15.16:0/3263943397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc0cfb80 msgr2=0x7f58cc15bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d14c6640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc0cfb80 0x7f58cc15bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d14c6640 1 -- 172.21.15.16:0/3263943397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc0b7850 con 0x7f58cc0b3a40 2024-09-12T01:34:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.686+0000 7f58d1cc7640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc161b00 0x7f58cc0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.687+0000 7f58d14c6640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0b3a40 0x7f58cc0a1640 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c430 tx=0x7f58c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.687+0000 7f58c1ffb640 1 -- 172.21.15.16:0/3263943397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c80074e0 con 0x7f58cc0b3a40 2024-09-12T01:34:05.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.687+0000 7f58c1ffb640 1 -- 172.21.15.16:0/3263943397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c80047e0 con 0x7f58cc0b3a40 2024-09-12T01:34:05.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.687+0000 7f58d372a640 1 -- 172.21.15.16:0/3263943397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc0b3f70 con 0x7f58cc0b3a40 2024-09-12T01:34:05.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.687+0000 7f58c1ffb640 1 -- 172.21.15.16:0/3263943397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8004bd0 con 0x7f58cc0b3a40 2024-09-12T01:34:05.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.687+0000 7f58d372a640 1 -- 172.21.15.16:0/3263943397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc0b4530 con 0x7f58cc0b3a40 2024-09-12T01:34:05.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.688+0000 7f58d372a640 1 -- 172.21.15.16:0/3263943397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58cc0d3030 con 0x7f58cc0b3a40 2024-09-12T01:34:05.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.688+0000 7f58c1ffb640 1 -- 172.21.15.16:0/3263943397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58c8011070 con 0x7f58cc0b3a40 2024-09-12T01:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.689+0000 7f58c1ffb640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f58b006be30 0x7f58b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.689+0000 7f58c1ffb640 1 -- 172.21.15.16:0/3263943397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5619+0+0 (secure 0 0 0) 0x7f58c808a9f0 con 0x7f58cc0b3a40 2024-09-12T01:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.689+0000 7f58d0cc5640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f58b006be30 0x7f58b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.690+0000 7f58d0cc5640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f58b006be30 0x7f58b006e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f58bc034000 tx=0x7f58bc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.694+0000 7f58c1ffb640 1 -- 172.21.15.16:0/3263943397 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58c8059480 con 0x7f58cc0b3a40 2024-09-12T01:34:05.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:05.868+0000 7f58d372a640 1 -- 172.21.15.16:0/3263943397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f58cc0d4b80 con 0x7f58cc0b3a40 2024-09-12T01:34:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.549+0000 7f58c1ffb640 1 -- 172.21.15.16:0/3263943397 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) ==== 170+0+0 (secure 0 0 0) 0x7f58c8058d80 con 0x7f58cc0b3a40 2024-09-12T01:34:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-12T01:34:06.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.553+0000 7f58d372a640 1 -- 172.21.15.16:0/3263943397 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f58b006be30 msgr2=0x7f58b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:06.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.553+0000 7f58d372a640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f58b006be30 0x7f58b006e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f58bc034000 tx=0x7f58bc02e000 comp rx=0 tx=0).stop 2024-09-12T01:34:06.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.553+0000 7f58d372a640 1 -- 172.21.15.16:0/3263943397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0b3a40 msgr2=0x7f58cc0a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:06.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.553+0000 7f58d372a640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0b3a40 0x7f58cc0a1640 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c430 tx=0x7f58c800c8f0 comp rx=0 tx=0).stop 2024-09-12T01:34:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.554+0000 7f58d372a640 1 -- 172.21.15.16:0/3263943397 shutdown_connections 2024-09-12T01:34:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.554+0000 7f58d372a640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f58b006be30 0x7f58b006e2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.554+0000 7f58d372a640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc161b00 0x7f58cc0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.554+0000 7f58d372a640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc0cfb80 0x7f58cc15bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.554+0000 7f58d372a640 1 --2- 172.21.15.16:0/3263943397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0b3a40 0x7f58cc0a1640 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.554+0000 7f58d372a640 1 -- 172.21.15.16:0/3263943397 >> 172.21.15.16:0/3263943397 conn(0x7f58cc08bfc0 msgr2=0x7f58cc0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.556+0000 7f58d372a640 1 -- 172.21.15.16:0/3263943397 shutdown_connections 2024-09-12T01:34:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.557+0000 7f58d372a640 1 -- 172.21.15.16:0/3263943397 wait complete. 2024-09-12T01:34:06.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T01:34:06.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.693+0000 7f702e12c640 1 Processor -- start 2024-09-12T01:34:06.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.693+0000 7f702e12c640 1 -- start start 2024-09-12T01:34:06.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.694+0000 7f702e12c640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f70280d0090 0x7f70280d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:06.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.694+0000 7f702e12c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70280d0e30 0x7f70280d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:06.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.694+0000 7f702e12c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70280cf050 con 0x7f702815e6a0 2024-09-12T01:34:06.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.694+0000 7f702e12c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f70280cfc00 con 0x7f70280d0e30 2024-09-12T01:34:06.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.694+0000 7f702e12c640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f70280cf2d0 con 0x7f70280d0090 2024-09-12T01:34:06.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.694+0000 7f70277fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f702815e6a0 0x7f70280d4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55064/0 (socket says 172.21.15.16:55064) 2024-09-12T01:34:06.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.694+0000 7f70277fe640 1 -- 172.21.15.16:0/2140986171 learned_addr learned my addr 172.21.15.16:0/2140986171 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:06.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.694+0000 7f7027fff640 1 --2- 172.21.15.16:0/2140986171 >> v2:172.21.15.16:3300/0 conn(0x7f70280d0e30 0x7f70280d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:06.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.694+0000 7f70267fc640 1 -- 172.21.15.16:0/2140986171 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569832756 0 0) 0x7f7018001290 con 0x7f702815e6a0 2024-09-12T01:34:06.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.694+0000 7f70267fc640 1 -- 172.21.15.16:0/2140986171 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7010003520 con 0x7f702815e6a0 2024-09-12T01:34:06.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.695+0000 7f7026ffd640 1 --2- 172.21.15.16:0/2140986171 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f70280d0090 0x7f70280d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:06.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.695+0000 7f7027fff640 1 -- 172.21.15.16:0/2140986171 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f70280d0090 msgr2=0x7f70280d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:06.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.695+0000 7f7027fff640 1 --2- 172.21.15.16:0/2140986171 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f70280d0090 0x7f70280d3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:06.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.695+0000 7f7027fff640 1 -- 172.21.15.16:0/2140986171 >> v1:172.21.15.16:6789/0 conn(0x7f702815e6a0 legacy=0x7f70280d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:06.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.695+0000 7f7027fff640 1 -- 172.21.15.16:0/2140986171 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70280cf5b0 con 0x7f70280d0e30 2024-09-12T01:34:06.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.695+0000 7f7026ffd640 1 --2- 172.21.15.16:0/2140986171 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f70280d0090 0x7f70280d3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:06.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.695+0000 7f7027fff640 1 --2- 172.21.15.16:0/2140986171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70280d0e30 0x7f70280d12e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7020009d90 tx=0x7f702002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144a2b06d1755b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:06.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.695+0000 7f70267fc640 1 -- 172.21.15.16:0/2140986171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020007740 con 0x7f70280d0e30 2024-09-12T01:34:06.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.695+0000 7f70267fc640 1 -- 172.21.15.16:0/2140986171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70200078e0 con 0x7f70280d0e30 2024-09-12T01:34:06.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.695+0000 7f70267fc640 1 -- 172.21.15.16:0/2140986171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020007cb0 con 0x7f70280d0e30 2024-09-12T01:34:06.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.696+0000 7f702e12c640 1 -- 172.21.15.16:0/2140986171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70280d0e30 msgr2=0x7f70280d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:06.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.696+0000 7f702e12c640 1 --2- 172.21.15.16:0/2140986171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70280d0e30 0x7f70280d12e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7020009d90 tx=0x7f702002a660 comp rx=0 tx=0).stop 2024-09-12T01:34:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.696+0000 7f702e12c640 1 -- 172.21.15.16:0/2140986171 shutdown_connections 2024-09-12T01:34:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.696+0000 7f702e12c640 1 --2- 172.21.15.16:0/2140986171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70280d0e30 0x7f70280d12e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.696+0000 7f702e12c640 1 --2- 172.21.15.16:0/2140986171 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f70280d0090 0x7f70280d3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.696+0000 7f702e12c640 1 -- 172.21.15.16:0/2140986171 >> 172.21.15.16:0/2140986171 conn(0x7f702809d730 msgr2=0x7f70280d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.696+0000 7f702e12c640 1 -- 172.21.15.16:0/2140986171 shutdown_connections 2024-09-12T01:34:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.696+0000 7f702e12c640 1 -- 172.21.15.16:0/2140986171 wait complete. 2024-09-12T01:34:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.697+0000 7f702e12c640 1 Processor -- start 2024-09-12T01:34:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.698+0000 7f702e12c640 1 -- start start 2024-09-12T01:34:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.698+0000 7f702e12c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70280d0090 0x7f70280cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.698+0000 7f702e12c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70280d0e30 0x7f70280cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.698+0000 7f702e12c640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f702815e6a0 0x7f70280cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.698+0000 7f702e12c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f70280b73f0 con 0x7f70280d0e30 2024-09-12T01:34:06.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.698+0000 7f702e12c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f70280b7270 con 0x7f70280d0090 2024-09-12T01:34:06.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.698+0000 7f702e12c640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f70280b7570 con 0x7f702815e6a0 2024-09-12T01:34:06.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.698+0000 7f7026ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70280d0e30 0x7f70280cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:06.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.698+0000 7f7026ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70280d0e30 0x7f70280cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41282/0 (socket says 172.21.15.16:41282) 2024-09-12T01:34:06.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.698+0000 7f7026ffd640 1 -- 172.21.15.16:0/3867378644 learned_addr learned my addr 172.21.15.16:0/3867378644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:06.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f7027fff640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f702815e6a0 0x7f70280cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:06.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f70277fe640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70280d0090 0x7f70280cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:06.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f7026ffd640 1 -- 172.21.15.16:0/3867378644 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f702815e6a0 msgr2=0x7f70280cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:06.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f7026ffd640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f702815e6a0 0x7f70280cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:06.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f7026ffd640 1 -- 172.21.15.16:0/3867378644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70280d0090 msgr2=0x7f70280cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:06.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f7026ffd640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70280d0090 0x7f70280cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:06.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f7026ffd640 1 -- 172.21.15.16:0/3867378644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70280cd210 con 0x7f70280d0e30 2024-09-12T01:34:06.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f7027fff640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f702815e6a0 0x7f70280cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:06.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f7026ffd640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70280d0e30 0x7f70280cd620 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f701c00c7f0 tx=0x7f701c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:06.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f700ffff640 1 -- 172.21.15.16:0/3867378644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c004230 con 0x7f70280d0e30 2024-09-12T01:34:06.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f700ffff640 1 -- 172.21.15.16:0/3867378644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701c0043d0 con 0x7f70280d0e30 2024-09-12T01:34:06.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f700ffff640 1 -- 172.21.15.16:0/3867378644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c007a70 con 0x7f70280d0e30 2024-09-12T01:34:06.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f702e12c640 1 -- 172.21.15.16:0/3867378644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70281f8180 con 0x7f70280d0e30 2024-09-12T01:34:06.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.699+0000 7f702e12c640 1 -- 172.21.15.16:0/3867378644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70281f8740 con 0x7f70280d0e30 2024-09-12T01:34:06.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.700+0000 7f702e12c640 1 -- 172.21.15.16:0/3867378644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70280d3160 con 0x7f70280d0e30 2024-09-12T01:34:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.701+0000 7f700ffff640 1 -- 172.21.15.16:0/3867378644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f701c007c10 con 0x7f70280d0e30 2024-09-12T01:34:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.701+0000 7f700ffff640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f701406be60 0x7f701406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.701+0000 7f700ffff640 1 -- 172.21.15.16:0/3867378644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6001+0+0 (secure 0 0 0) 0x7f701c08aff0 con 0x7f70280d0e30 2024-09-12T01:34:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.701+0000 7f70277fe640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f701406be60 0x7f701406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:06.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.702+0000 7f70277fe640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f701406be60 0x7f701406e320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7018001530 tx=0x7f7018009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:06.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.706+0000 7f700ffff640 1 -- 172.21.15.16:0/3867378644 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f701c015020 con 0x7f70280d0e30 2024-09-12T01:34:06.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:06.917+0000 7f702e12c640 1 -- 172.21.15.16:0/3867378644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f702806d9a0 con 0x7f70280d0e30 2024-09-12T01:34:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.551+0000 7f700ffff640 1 -- 172.21.15.16:0/3867378644 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) ==== 172+0+0 (secure 0 0 0) 0x7f701c059860 con 0x7f70280d0e30 2024-09-12T01:34:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 5 and data pool 6 2024-09-12T01:34:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.557+0000 7f702e12c640 1 -- 172.21.15.16:0/3867378644 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f701406be60 msgr2=0x7f701406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.557+0000 7f702e12c640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f701406be60 0x7f701406e320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7018001530 tx=0x7f7018009040 comp rx=0 tx=0).stop 2024-09-12T01:34:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.557+0000 7f702e12c640 1 -- 172.21.15.16:0/3867378644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70280d0e30 msgr2=0x7f70280cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.557+0000 7f702e12c640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70280d0e30 0x7f70280cd620 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f701c00c7f0 tx=0x7f701c00ccb0 comp rx=0 tx=0).stop 2024-09-12T01:34:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.558+0000 7f702e12c640 1 -- 172.21.15.16:0/3867378644 shutdown_connections 2024-09-12T01:34:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.558+0000 7f702e12c640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f701406be60 0x7f701406e320 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7018001530 tx=0x7f7018009040 comp rx=0 tx=0).stop 2024-09-12T01:34:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.558+0000 7f702e12c640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f702815e6a0 0x7f70280cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.558+0000 7f702e12c640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70280d0e30 0x7f70280cd620 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.558+0000 7f702e12c640 1 --2- 172.21.15.16:0/3867378644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70280d0090 0x7f70280cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.558+0000 7f702e12c640 1 -- 172.21.15.16:0/3867378644 >> 172.21.15.16:0/3867378644 conn(0x7f702809d730 msgr2=0x7f70280d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.564+0000 7f702e12c640 1 -- 172.21.15.16:0/3867378644 shutdown_connections 2024-09-12T01:34:07.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.568+0000 7f702e12c640 1 -- 172.21.15.16:0/3867378644 wait complete. 2024-09-12T01:34:07.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:34:07.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.704+0000 7ff6c858d640 1 Processor -- start 2024-09-12T01:34:07.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.704+0000 7ff6c858d640 1 -- start start 2024-09-12T01:34:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.705+0000 7ff6c858d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff6c00d1550 0x7ff6c01691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.705+0000 7ff6c858d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6c00d3480 con 0x7ff6c00ceee0 2024-09-12T01:34:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.705+0000 7ff6c858d640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6c00d3760 con 0x7ff6c015e560 2024-09-12T01:34:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.705+0000 7ff6c858d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff6c00d3a40 con 0x7ff6c00d1550 2024-09-12T01:34:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.706+0000 7ff6c5b28640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff6c00ceee0 0x7ff6c00d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55074/0 (socket says 172.21.15.16:55074) 2024-09-12T01:34:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.706+0000 7ff6c5b28640 1 -- 172.21.15.16:0/4122800376 learned_addr learned my addr 172.21.15.16:0/4122800376 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.706+0000 7ff6c6b2a640 1 --2- 172.21.15.16:0/4122800376 >> v2:172.21.15.16:3300/0 conn(0x7ff6c00d1550 0x7ff6c01691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.706+0000 7ff6c5327640 1 -- 172.21.15.16:0/4122800376 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1226272430 0 0) 0x7ff6c00d3480 con 0x7ff6c00ceee0 2024-09-12T01:34:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.706+0000 7ff6c5327640 1 -- 172.21.15.16:0/4122800376 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6a8003520 con 0x7ff6c00ceee0 2024-09-12T01:34:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.706+0000 7ff6c6b2a640 1 -- 172.21.15.16:0/4122800376 >> v1:172.21.15.33:6789/0 conn(0x7ff6c015e560 legacy=0x7ff6c00cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.706+0000 7ff6c6b2a640 1 -- 172.21.15.16:0/4122800376 >> v1:172.21.15.16:6789/0 conn(0x7ff6c00ceee0 legacy=0x7ff6c00d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.706+0000 7ff6c6b2a640 1 -- 172.21.15.16:0/4122800376 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c00d3d20 con 0x7ff6c00d1550 2024-09-12T01:34:07.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.707+0000 7ff6c6b2a640 1 --2- 172.21.15.16:0/4122800376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c00d1550 0x7ff6c01691c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80097a0 tx=0x7ff6b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141710931c2659c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:07.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.707+0000 7ff6c5327640 1 -- 172.21.15.16:0/4122800376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8037070 con 0x7ff6c00d1550 2024-09-12T01:34:07.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.707+0000 7ff6c5327640 1 -- 172.21.15.16:0/4122800376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b8032440 con 0x7ff6c00d1550 2024-09-12T01:34:07.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.707+0000 7ff6c5327640 1 -- 172.21.15.16:0/4122800376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8032810 con 0x7ff6c00d1550 2024-09-12T01:34:07.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.707+0000 7ff6c858d640 1 -- 172.21.15.16:0/4122800376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c00d1550 msgr2=0x7ff6c01691c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:07.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.707+0000 7ff6c858d640 1 --2- 172.21.15.16:0/4122800376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c00d1550 0x7ff6c01691c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80097a0 tx=0x7ff6b802a230 comp rx=0 tx=0).stop 2024-09-12T01:34:07.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.708+0000 7ff6c858d640 1 -- 172.21.15.16:0/4122800376 shutdown_connections 2024-09-12T01:34:07.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.708+0000 7ff6c858d640 1 --2- 172.21.15.16:0/4122800376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c00d1550 0x7ff6c01691c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:07.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.708+0000 7ff6c858d640 1 -- 172.21.15.16:0/4122800376 >> 172.21.15.16:0/4122800376 conn(0x7ff6c0073850 msgr2=0x7ff6c006cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:07.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.708+0000 7ff6c858d640 1 -- 172.21.15.16:0/4122800376 shutdown_connections 2024-09-12T01:34:07.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.708+0000 7ff6c858d640 1 -- 172.21.15.16:0/4122800376 wait complete. 2024-09-12T01:34:07.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.709+0000 7ff6c858d640 1 Processor -- start 2024-09-12T01:34:07.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.709+0000 7ff6c858d640 1 -- start start 2024-09-12T01:34:07.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.709+0000 7ff6c858d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff6c00ceee0 0x7ff6c0161460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c858d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c00d1550 0x7ff6c014d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c858d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c015e560 0x7ff6c01f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c858d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c015eac0 con 0x7ff6c00d1550 2024-09-12T01:34:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c858d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c015e940 con 0x7ff6c015e560 2024-09-12T01:34:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c858d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c00b87a0 con 0x7ff6c00ceee0 2024-09-12T01:34:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c5b28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c00d1550 0x7ff6c014d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c5b28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c00d1550 0x7ff6c014d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41304/0 (socket says 172.21.15.16:41304) 2024-09-12T01:34:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c5b28640 1 -- 172.21.15.16:0/925270921 learned_addr learned my addr 172.21.15.16:0/925270921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c6b2a640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c015e560 0x7ff6c01f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c6329640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff6c00ceee0 0x7ff6c0161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c5b28640 1 -- 172.21.15.16:0/925270921 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff6c00ceee0 msgr2=0x7ff6c0161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c5b28640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff6c00ceee0 0x7ff6c0161460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c5b28640 1 -- 172.21.15.16:0/925270921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c015e560 msgr2=0x7ff6c01f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c5b28640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c015e560 0x7ff6c01f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c5b28640 1 -- 172.21.15.16:0/925270921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c014de90 con 0x7ff6c00d1550 2024-09-12T01:34:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c6b2a640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c015e560 0x7ff6c01f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c5b28640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c00d1550 0x7ff6c014d950 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400c530 tx=0x7ff6b400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6beffd640 1 -- 172.21.15.16:0/925270921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4017070 con 0x7ff6c00d1550 2024-09-12T01:34:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6beffd640 1 -- 172.21.15.16:0/925270921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b4005ce0 con 0x7ff6c00d1550 2024-09-12T01:34:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6beffd640 1 -- 172.21.15.16:0/925270921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40072b0 con 0x7ff6c00d1550 2024-09-12T01:34:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c858d640 1 -- 172.21.15.16:0/925270921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c00cb7d0 con 0x7ff6c00d1550 2024-09-12T01:34:07.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.710+0000 7ff6c858d640 1 -- 172.21.15.16:0/925270921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c00cbd90 con 0x7ff6c00d1550 2024-09-12T01:34:07.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.711+0000 7ff6c858d640 1 -- 172.21.15.16:0/925270921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c00cb560 con 0x7ff6c00d1550 2024-09-12T01:34:07.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.712+0000 7ff6beffd640 1 -- 172.21.15.16:0/925270921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6b401a070 con 0x7ff6c00d1550 2024-09-12T01:34:07.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.712+0000 7ff6beffd640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff6ac06be60 0x7ff6ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:07.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.712+0000 7ff6beffd640 1 -- 172.21.15.16:0/925270921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ff6b408a3b0 con 0x7ff6c00d1550 2024-09-12T01:34:07.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.713+0000 7ff6c6329640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff6ac06be60 0x7ff6ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:07.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.713+0000 7ff6c6329640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff6ac06be60 0x7ff6ac06e320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0001ca0 tx=0x7ff6b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:07.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.717+0000 7ff6beffd640 1 -- 172.21.15.16:0/925270921 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff6b4058c90 con 0x7ff6c00d1550 2024-09-12T01:34:07.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.893+0000 7ff6c858d640 1 -- 172.21.15.16:0/925270921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6c00d0e40 con 0x7ff6c00d1550 2024-09-12T01:34:07.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.894+0000 7ff6beffd640 1 -- 172.21.15.16:0/925270921 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16991 (secure 0 0 0) 0x7ff6b4058590 con 0x7ff6c00d1550 2024-09-12T01:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:34:06.918579+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:34:04.823039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-12T01:34:05.900693+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:34:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.896+0000 7ff6c858d640 1 -- 172.21.15.16:0/925270921 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff6ac06be60 msgr2=0x7ff6ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.896+0000 7ff6c858d640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff6ac06be60 0x7ff6ac06e320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0001ca0 tx=0x7ff6b0009040 comp rx=0 tx=0).stop 2024-09-12T01:34:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.896+0000 7ff6c858d640 1 -- 172.21.15.16:0/925270921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c00d1550 msgr2=0x7ff6c014d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.896+0000 7ff6c858d640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c00d1550 0x7ff6c014d950 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400c530 tx=0x7ff6b400c9f0 comp rx=0 tx=0).stop 2024-09-12T01:34:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.896+0000 7ff6c858d640 1 -- 172.21.15.16:0/925270921 shutdown_connections 2024-09-12T01:34:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.896+0000 7ff6c858d640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff6ac06be60 0x7ff6ac06e320 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.896+0000 7ff6c858d640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c015e560 0x7ff6c01f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.896+0000 7ff6c858d640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6c00d1550 0x7ff6c014d950 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.896+0000 7ff6c858d640 1 --2- 172.21.15.16:0/925270921 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff6c00ceee0 0x7ff6c0161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.896+0000 7ff6c858d640 1 -- 172.21.15.16:0/925270921 >> 172.21.15.16:0/925270921 conn(0x7ff6c0073850 msgr2=0x7ff6c00b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.896+0000 7ff6c858d640 1 -- 172.21.15.16:0/925270921 shutdown_connections 2024-09-12T01:34:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:07.896+0000 7ff6c858d640 1 -- 172.21.15.16:0/925270921 wait complete. 2024-09-12T01:34:07.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:34:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.009+0000 7f99e46c9640 1 Processor -- start 2024-09-12T01:34:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.009+0000 7f99e46c9640 1 -- start start 2024-09-12T01:34:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.010+0000 7f99e46c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99dc15e510 0x7f99dc0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.010+0000 7f99e46c9640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f99dc0cf040 0x7f99dc0d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.010+0000 7f99e46c9640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99dc0d3430 con 0x7f99dc0d1200 2024-09-12T01:34:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.010+0000 7f99e46c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f99dc0d1670 con 0x7f99dc15e510 2024-09-12T01:34:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.010+0000 7f99e46c9640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f99dc0d36d0 con 0x7f99dc0cf040 2024-09-12T01:34:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.010+0000 7f99e2465640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99dc15e510 0x7f99dc0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.010+0000 7f99e2465640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99dc15e510 0x7f99dc0ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41318/0 (socket says 172.21.15.16:41318) 2024-09-12T01:34:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.010+0000 7f99e2465640 1 -- 172.21.15.16:0/2522080577 learned_addr learned my addr 172.21.15.16:0/2522080577 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.011+0000 7f99e1c64640 1 --2- 172.21.15.16:0/2522080577 >> v2:172.21.15.33:3300/0 conn(0x7f99dc0cf040 0x7f99dc0d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.011+0000 7f99e2465640 1 -- 172.21.15.16:0/2522080577 >> v2:172.21.15.33:3300/0 conn(0x7f99dc0cf040 msgr2=0x7f99dc0d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:08.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.011+0000 7f99e2465640 1 --2- 172.21.15.16:0/2522080577 >> v2:172.21.15.33:3300/0 conn(0x7f99dc0cf040 0x7f99dc0d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.011+0000 7f99e2465640 1 -- 172.21.15.16:0/2522080577 >> v1:172.21.15.33:6789/0 conn(0x7f99dc0d1200 legacy=0x7f99dc0b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:08.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.011+0000 7f99e2465640 1 -- 172.21.15.16:0/2522080577 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc0d39b0 con 0x7f99dc15e510 2024-09-12T01:34:08.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.011+0000 7f99e2465640 1 --2- 172.21.15.16:0/2522080577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99dc15e510 0x7f99dc0ceb00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0d4460 tx=0x7f99cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cfb39198eea7b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.012+0000 7f99e1463640 1 -- 172.21.15.16:0/2522080577 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc036070 con 0x7f99dc15e510 2024-09-12T01:34:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.012+0000 7f99e1463640 1 -- 172.21.15.16:0/2522080577 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99cc031600 con 0x7f99dc15e510 2024-09-12T01:34:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.012+0000 7f99e1463640 1 -- 172.21.15.16:0/2522080577 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0319d0 con 0x7f99dc15e510 2024-09-12T01:34:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.012+0000 7f99e46c9640 1 -- 172.21.15.16:0/2522080577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99dc15e510 msgr2=0x7f99dc0ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.012+0000 7f99e46c9640 1 --2- 172.21.15.16:0/2522080577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99dc15e510 0x7f99dc0ceb00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0d4460 tx=0x7f99cc02a6a0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.012+0000 7f99e46c9640 1 -- 172.21.15.16:0/2522080577 shutdown_connections 2024-09-12T01:34:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.012+0000 7f99e46c9640 1 --2- 172.21.15.16:0/2522080577 >> v2:172.21.15.33:3300/0 conn(0x7f99dc0cf040 0x7f99dc0d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.012+0000 7f99e46c9640 1 --2- 172.21.15.16:0/2522080577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99dc15e510 0x7f99dc0ceb00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.012+0000 7f99e46c9640 1 -- 172.21.15.16:0/2522080577 >> 172.21.15.16:0/2522080577 conn(0x7f99dc06d1a0 msgr2=0x7f99dc073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:08.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.012+0000 7f99e46c9640 1 -- 172.21.15.16:0/2522080577 shutdown_connections 2024-09-12T01:34:08.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.012+0000 7f99e46c9640 1 -- 172.21.15.16:0/2522080577 wait complete. 2024-09-12T01:34:08.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.014+0000 7f99e46c9640 1 Processor -- start 2024-09-12T01:34:08.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.014+0000 7f99e46c9640 1 -- start start 2024-09-12T01:34:08.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.015+0000 7f99e46c9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99dc0cf040 0x7f99dc0ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:08.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.015+0000 7f99e46c9640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f99dc0d1200 0x7f99dc160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:08.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.015+0000 7f99e46c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99dc15e510 0x7f99dc158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:08.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.015+0000 7f99e46c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc0d22a0 con 0x7f99dc15e510 2024-09-12T01:34:08.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.015+0000 7f99e46c9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc0d3f70 con 0x7f99dc0cf040 2024-09-12T01:34:08.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.015+0000 7f99e46c9640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f99dc15e8f0 con 0x7f99dc0d1200 2024-09-12T01:34:08.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.015+0000 7f99e2c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99dc15e510 0x7f99dc158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:08.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.015+0000 7f99e2c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99dc15e510 0x7f99dc158cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41334/0 (socket says 172.21.15.16:41334) 2024-09-12T01:34:08.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.015+0000 7f99e2c66640 1 -- 172.21.15.16:0/3643599122 learned_addr learned my addr 172.21.15.16:0/3643599122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:08.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.015+0000 7f99e2465640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99dc0cf040 0x7f99dc0ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:08.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.015+0000 7f99e1c64640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f99dc0d1200 0x7f99dc160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:08.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.016+0000 7f99e2c66640 1 -- 172.21.15.16:0/3643599122 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f99dc0d1200 msgr2=0x7f99dc160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:08.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.016+0000 7f99e2c66640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f99dc0d1200 0x7f99dc160d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.016+0000 7f99e2c66640 1 -- 172.21.15.16:0/3643599122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99dc0cf040 msgr2=0x7f99dc0ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:08.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.016+0000 7f99e2c66640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99dc0cf040 0x7f99dc0ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.016+0000 7f99e2c66640 1 -- 172.21.15.16:0/3643599122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc15fc80 con 0x7f99dc15e510 2024-09-12T01:34:08.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.016+0000 7f99e2c66640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99dc15e510 0x7f99dc158cc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f99d4001eb0 tx=0x7f99d400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:08.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.016+0000 7f99daffd640 1 -- 172.21.15.16:0/3643599122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4017070 con 0x7f99dc15e510 2024-09-12T01:34:08.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.016+0000 7f99daffd640 1 -- 172.21.15.16:0/3643599122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d40052d0 con 0x7f99dc15e510 2024-09-12T01:34:08.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.016+0000 7f99e46c9640 1 -- 172.21.15.16:0/3643599122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99dc15ff60 con 0x7f99dc15e510 2024-09-12T01:34:08.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.016+0000 7f99daffd640 1 -- 172.21.15.16:0/3643599122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4003810 con 0x7f99dc15e510 2024-09-12T01:34:08.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.016+0000 7f99e46c9640 1 -- 172.21.15.16:0/3643599122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99dc160520 con 0x7f99dc15e510 2024-09-12T01:34:08.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.017+0000 7f99e46c9640 1 -- 172.21.15.16:0/3643599122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99dc0d0cc0 con 0x7f99dc15e510 2024-09-12T01:34:08.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.018+0000 7f99daffd640 1 -- 172.21.15.16:0/3643599122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f99d401a070 con 0x7f99dc15e510 2024-09-12T01:34:08.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.019+0000 7f99daffd640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f99c806be30 0x7f99c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:08.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.019+0000 7f99e2465640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f99c806be30 0x7f99c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:08.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.019+0000 7f99daffd640 1 -- 172.21.15.16:0/3643599122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f99d408a350 con 0x7f99dc15e510 2024-09-12T01:34:08.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.019+0000 7f99e2465640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f99c806be30 0x7f99c806e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f99cc033000 tx=0x7f99cc009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:08.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.023+0000 7f99daffd640 1 -- 172.21.15.16:0/3643599122 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f99d4058490 con 0x7f99dc15e510 2024-09-12T01:34:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.200+0000 7f99e46c9640 1 -- 172.21.15.16:0/3643599122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99dc0cfaf0 con 0x7f99dc15e510 2024-09-12T01:34:08.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.201+0000 7f99daffd640 1 -- 172.21.15.16:0/3643599122 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16991 (secure 0 0 0) 0x7f99d405c790 con 0x7f99dc15e510 2024-09-12T01:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:34:06.918579+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:34:04.823039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-12T01:34:05.900693+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:34:08.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.203+0000 7f99e46c9640 1 -- 172.21.15.16:0/3643599122 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f99c806be30 msgr2=0x7f99c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:08.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.203+0000 7f99e46c9640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f99c806be30 0x7f99c806e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f99cc033000 tx=0x7f99cc009000 comp rx=0 tx=0).stop 2024-09-12T01:34:08.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.203+0000 7f99e46c9640 1 -- 172.21.15.16:0/3643599122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99dc15e510 msgr2=0x7f99dc158cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:08.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.203+0000 7f99e46c9640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99dc15e510 0x7f99dc158cc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f99d4001eb0 tx=0x7f99d400c890 comp rx=0 tx=0).stop 2024-09-12T01:34:08.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.203+0000 7f99e46c9640 1 -- 172.21.15.16:0/3643599122 shutdown_connections 2024-09-12T01:34:08.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.203+0000 7f99e46c9640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f99c806be30 0x7f99c806e2f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.203+0000 7f99e46c9640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99dc15e510 0x7f99dc158cc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.203+0000 7f99e46c9640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f99dc0d1200 0x7f99dc160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.203+0000 7f99e46c9640 1 --2- 172.21.15.16:0/3643599122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99dc0cf040 0x7f99dc0ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.203+0000 7f99e46c9640 1 -- 172.21.15.16:0/3643599122 >> 172.21.15.16:0/3643599122 conn(0x7f99dc06d1a0 msgr2=0x7f99dc0b7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:08.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.203+0000 7f99e46c9640 1 -- 172.21.15.16:0/3643599122 shutdown_connections 2024-09-12T01:34:08.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.204+0000 7f99e46c9640 1 -- 172.21.15.16:0/3643599122 wait complete. 2024-09-12T01:34:08.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T01:34:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.321+0000 7f3621027640 1 Processor -- start 2024-09-12T01:34:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.321+0000 7f3621027640 1 -- start start 2024-09-12T01:34:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.321+0000 7f3621027640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f361c15e940 0x7f361c1692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.321+0000 7f3621027640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f361c072180 con 0x7f361c1589c0 2024-09-12T01:34:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.322+0000 7f3621027640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f361c072420 con 0x7f361c0cda30 2024-09-12T01:34:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.322+0000 7f3621027640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f361c0ca880 con 0x7f361c15e940 2024-09-12T01:34:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.322+0000 7f361ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f361c1589c0 0x7f361c0cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55080/0 (socket says 172.21.15.16:55080) 2024-09-12T01:34:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.322+0000 7f361ad9d640 1 -- 172.21.15.16:0/1373277004 learned_addr learned my addr 172.21.15.16:0/1373277004 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.322+0000 7f361b59e640 1 --2- 172.21.15.16:0/1373277004 >> v2:172.21.15.16:3300/0 conn(0x7f361c15e940 0x7f361c1692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.322+0000 7f3619d9b640 1 -- 172.21.15.16:0/1373277004 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 160161942 0 0) 0x7f361c072180 con 0x7f361c1589c0 2024-09-12T01:34:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.322+0000 7f3619d9b640 1 -- 172.21.15.16:0/1373277004 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3604003520 con 0x7f361c1589c0 2024-09-12T01:34:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.322+0000 7f361b59e640 1 -- 172.21.15.16:0/1373277004 >> v1:172.21.15.33:6789/0 conn(0x7f361c0cda30 legacy=0x7f361c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.322+0000 7f361b59e640 1 -- 172.21.15.16:0/1373277004 >> v1:172.21.15.16:6789/0 conn(0x7f361c1589c0 legacy=0x7f361c0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.322+0000 7f361b59e640 1 -- 172.21.15.16:0/1373277004 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c072700 con 0x7f361c15e940 2024-09-12T01:34:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.323+0000 7f361b59e640 1 --2- 172.21.15.16:0/1373277004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c15e940 0x7f361c1692c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3614009d90 tx=0x7f361402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d4064b8e84356e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.323+0000 7f3619d9b640 1 -- 172.21.15.16:0/1373277004 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614007740 con 0x7f361c15e940 2024-09-12T01:34:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.323+0000 7f3619d9b640 1 -- 172.21.15.16:0/1373277004 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36140078e0 con 0x7f361c15e940 2024-09-12T01:34:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.323+0000 7f3619d9b640 1 -- 172.21.15.16:0/1373277004 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614007cb0 con 0x7f361c15e940 2024-09-12T01:34:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.323+0000 7f3621027640 1 -- 172.21.15.16:0/1373277004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c15e940 msgr2=0x7f361c1692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.323+0000 7f3621027640 1 --2- 172.21.15.16:0/1373277004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c15e940 0x7f361c1692c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3614009d90 tx=0x7f361402a6f0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.323+0000 7f3621027640 1 -- 172.21.15.16:0/1373277004 shutdown_connections 2024-09-12T01:34:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.324+0000 7f3621027640 1 --2- 172.21.15.16:0/1373277004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c15e940 0x7f361c1692c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.324+0000 7f3621027640 1 -- 172.21.15.16:0/1373277004 >> 172.21.15.16:0/1373277004 conn(0x7f361c0ccf10 msgr2=0x7f361c0cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.324+0000 7f3621027640 1 -- 172.21.15.16:0/1373277004 shutdown_connections 2024-09-12T01:34:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.324+0000 7f3621027640 1 -- 172.21.15.16:0/1373277004 wait complete. 2024-09-12T01:34:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.325+0000 7f3621027640 1 Processor -- start 2024-09-12T01:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.325+0000 7f3621027640 1 -- start start 2024-09-12T01:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f3621027640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c0cda30 0x7f361c15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f3621027640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f361c1589c0 0x7f361c1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f3621027640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f361c15e940 0x7f361c157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f3621027640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f361c158da0 con 0x7f361c0cda30 2024-09-12T01:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f3621027640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f361c072cc0 con 0x7f361c1589c0 2024-09-12T01:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f3621027640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f361c0caae0 con 0x7f361c15e940 2024-09-12T01:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f361ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c0cda30 0x7f361c15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f361ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c0cda30 0x7f361c15cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41348/0 (socket says 172.21.15.16:41348) 2024-09-12T01:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f361ad9d640 1 -- 172.21.15.16:0/4065980695 learned_addr learned my addr 172.21.15.16:0/4065980695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f361a59c640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f361c1589c0 0x7f361c1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f361b59e640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f361c15e940 0x7f361c157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f361ad9d640 1 -- 172.21.15.16:0/4065980695 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f361c15e940 msgr2=0x7f361c157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f361ad9d640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f361c15e940 0x7f361c157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f361ad9d640 1 -- 172.21.15.16:0/4065980695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f361c1589c0 msgr2=0x7f361c1579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f361ad9d640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f361c1589c0 0x7f361c1579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f361ad9d640 1 -- 172.21.15.16:0/4065980695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c1f4050 con 0x7f361c0cda30 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.326+0000 7f361ad9d640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c0cda30 0x7f361c15cff0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f360c001eb0 tx=0x7f360c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.327+0000 7f36037fe640 1 -- 172.21.15.16:0/4065980695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c017070 con 0x7f361c0cda30 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.327+0000 7f36037fe640 1 -- 172.21.15.16:0/4065980695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c005ce0 con 0x7f361c0cda30 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.327+0000 7f3621027640 1 -- 172.21.15.16:0/4065980695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361c1f42f0 con 0x7f361c0cda30 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.327+0000 7f36037fe640 1 -- 172.21.15.16:0/4065980695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c007270 con 0x7f361c0cda30 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.327+0000 7f3621027640 1 -- 172.21.15.16:0/4065980695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361c1f4dd0 con 0x7f361c0cda30 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.328+0000 7f3621027640 1 -- 172.21.15.16:0/4065980695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f361c0cfcf0 con 0x7f361c0cda30 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.328+0000 7f36037fe640 1 -- 172.21.15.16:0/4065980695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f360c01a070 con 0x7f361c0cda30 2024-09-12T01:34:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.329+0000 7f36037fe640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f360806be30 0x7f360806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.329+0000 7f36037fe640 1 -- 172.21.15.16:0/4065980695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f360c08a240 con 0x7f361c0cda30 2024-09-12T01:34:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.329+0000 7f361a59c640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f360806be30 0x7f360806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.329+0000 7f361a59c640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f360806be30 0x7f360806e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f361c0ce160 tx=0x7f3610009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.333+0000 7f36037fe640 1 -- 172.21.15.16:0/4065980695 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f360c058410 con 0x7f361c0cda30 2024-09-12T01:34:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:08.547+0000 7f3621027640 1 -- 172.21.15.16:0/4065980695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f361c0731a0 con 0x7f361c0cda30 2024-09-12T01:34:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.018+0000 7f36037fe640 1 -- 172.21.15.16:0/4065980695 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f360c05c710 con 0x7f361c0cda30 2024-09-12T01:34:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.021+0000 7f3621027640 1 -- 172.21.15.16:0/4065980695 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f360806be30 msgr2=0x7f360806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.021+0000 7f3621027640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f360806be30 0x7f360806e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f361c0ce160 tx=0x7f3610009040 comp rx=0 tx=0).stop 2024-09-12T01:34:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.021+0000 7f3621027640 1 -- 172.21.15.16:0/4065980695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c0cda30 msgr2=0x7f361c15cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.021+0000 7f3621027640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c0cda30 0x7f361c15cff0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f360c001eb0 tx=0x7f360c00c960 comp rx=0 tx=0).stop 2024-09-12T01:34:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.021+0000 7f3621027640 1 -- 172.21.15.16:0/4065980695 shutdown_connections 2024-09-12T01:34:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.021+0000 7f3621027640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f360806be30 0x7f360806e2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.021+0000 7f3621027640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f361c15e940 0x7f361c157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.021+0000 7f3621027640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f361c1589c0 0x7f361c1579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.021+0000 7f3621027640 1 --2- 172.21.15.16:0/4065980695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c0cda30 0x7f361c15cff0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.021+0000 7f3621027640 1 -- 172.21.15.16:0/4065980695 >> 172.21.15.16:0/4065980695 conn(0x7f361c0ccf10 msgr2=0x7f361c1ea090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.021+0000 7f3621027640 1 -- 172.21.15.16:0/4065980695 shutdown_connections 2024-09-12T01:34:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.021+0000 7f3621027640 1 -- 172.21.15.16:0/4065980695 wait complete. 2024-09-12T01:34:09.034 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T01:34:09.034 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T01:34:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.146+0000 7f78524a4640 1 Processor -- start 2024-09-12T01:34:09.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.146+0000 7f78524a4640 1 -- start start 2024-09-12T01:34:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f78524a4640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f784c15e6a0 0x7f784c0d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f78524a4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f784c0d3160 0x7f784c0d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f78524a4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f784c0d0e30 0x7f784c0d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f78524a4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f784c0cfc00 con 0x7f784c0d0e30 2024-09-12T01:34:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f78524a4640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f784c0cf050 con 0x7f784c15e6a0 2024-09-12T01:34:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f78524a4640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f784c0cf2d0 con 0x7f784c0d3160 2024-09-12T01:34:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f7850a41640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f784c0d0e30 0x7f784c0d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f7850a41640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f784c0d0e30 0x7f784c0d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41352/0 (socket says 172.21.15.16:41352) 2024-09-12T01:34:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f7850a41640 1 -- 172.21.15.16:0/121456340 learned_addr learned my addr 172.21.15.16:0/121456340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f784bfff640 1 --2- 172.21.15.16:0/121456340 >> v2:172.21.15.33:3300/0 conn(0x7f784c15e6a0 0x7f784c0d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f784b7fe640 1 --2- 172.21.15.16:0/121456340 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f784c0d3160 0x7f784c0d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f7850a41640 1 -- 172.21.15.16:0/121456340 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f784c0d3160 msgr2=0x7f784c0d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f7850a41640 1 --2- 172.21.15.16:0/121456340 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f784c0d3160 0x7f784c0d4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f7850a41640 1 -- 172.21.15.16:0/121456340 >> v2:172.21.15.33:3300/0 conn(0x7f784c15e6a0 msgr2=0x7f784c0d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f7850a41640 1 --2- 172.21.15.16:0/121456340 >> v2:172.21.15.33:3300/0 conn(0x7f784c15e6a0 0x7f784c0d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f7850a41640 1 -- 172.21.15.16:0/121456340 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c0cf5b0 con 0x7f784c0d0e30 2024-09-12T01:34:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f784bfff640 1 --2- 172.21.15.16:0/121456340 >> v2:172.21.15.33:3300/0 conn(0x7f784c15e6a0 0x7f784c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.147+0000 7f784b7fe640 1 --2- 172.21.15.16:0/121456340 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f784c0d3160 0x7f784c0d4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.148+0000 7f7850a41640 1 --2- 172.21.15.16:0/121456340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784c0d0e30 0x7f784c0d12e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f784c08ea00 tx=0x7f784402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b458599cd7680490 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.148+0000 7f784affd640 1 -- 172.21.15.16:0/121456340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844036070 con 0x7f784c0d0e30 2024-09-12T01:34:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.148+0000 7f784affd640 1 -- 172.21.15.16:0/121456340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7844031600 con 0x7f784c0d0e30 2024-09-12T01:34:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.148+0000 7f784affd640 1 -- 172.21.15.16:0/121456340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440319d0 con 0x7f784c0d0e30 2024-09-12T01:34:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.149+0000 7f78524a4640 1 -- 172.21.15.16:0/121456340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784c0d0e30 msgr2=0x7f784c0d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.149+0000 7f78524a4640 1 --2- 172.21.15.16:0/121456340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784c0d0e30 0x7f784c0d12e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f784c08ea00 tx=0x7f784402a6a0 comp rx=0 tx=0).stop 2024-09-12T01:34:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.149+0000 7f78524a4640 1 -- 172.21.15.16:0/121456340 shutdown_connections 2024-09-12T01:34:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.149+0000 7f78524a4640 1 --2- 172.21.15.16:0/121456340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784c0d0e30 0x7f784c0d12e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.149+0000 7f78524a4640 1 --2- 172.21.15.16:0/121456340 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f784c0d3160 0x7f784c0d4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.149+0000 7f78524a4640 1 --2- 172.21.15.16:0/121456340 >> v2:172.21.15.33:3300/0 conn(0x7f784c15e6a0 0x7f784c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.149+0000 7f78524a4640 1 -- 172.21.15.16:0/121456340 >> 172.21.15.16:0/121456340 conn(0x7f784c09d730 msgr2=0x7f784c0d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.149+0000 7f78524a4640 1 -- 172.21.15.16:0/121456340 shutdown_connections 2024-09-12T01:34:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.149+0000 7f78524a4640 1 -- 172.21.15.16:0/121456340 wait complete. 2024-09-12T01:34:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.150+0000 7f78524a4640 1 Processor -- start 2024-09-12T01:34:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.150+0000 7f78524a4640 1 -- start start 2024-09-12T01:34:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f78524a4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f784c0d0e30 0x7f784c157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f78524a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784c0d3160 0x7f784c155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f78524a4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f784c15e6a0 0x7f784c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f78524a4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f784c0d51f0 con 0x7f784c0d3160 2024-09-12T01:34:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f78524a4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f784c0d5070 con 0x7f784c0d0e30 2024-09-12T01:34:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f78524a4640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f784c0b7270 con 0x7f784c15e6a0 2024-09-12T01:34:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f784b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784c0d3160 0x7f784c155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f784b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784c0d3160 0x7f784c155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41368/0 (socket says 172.21.15.16:41368) 2024-09-12T01:34:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f784b7fe640 1 -- 172.21.15.16:0/301298166 learned_addr learned my addr 172.21.15.16:0/301298166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f784bfff640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f784c0d0e30 0x7f784c157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f7850a41640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f784c15e6a0 0x7f784c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f784b7fe640 1 -- 172.21.15.16:0/301298166 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f784c15e6a0 msgr2=0x7f784c0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f784b7fe640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f784c15e6a0 0x7f784c0c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f784b7fe640 1 -- 172.21.15.16:0/301298166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f784c0d0e30 msgr2=0x7f784c157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f784b7fe640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f784c0d0e30 0x7f784c157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.151+0000 7f784b7fe640 1 -- 172.21.15.16:0/301298166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c1559f0 con 0x7f784c0d3160 2024-09-12T01:34:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.152+0000 7f784b7fe640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784c0d3160 0x7f784c155250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f784000c7f0 tx=0x7f784000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.152+0000 7f7848ff9640 1 -- 172.21.15.16:0/301298166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840004230 con 0x7f784c0d3160 2024-09-12T01:34:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.152+0000 7f7848ff9640 1 -- 172.21.15.16:0/301298166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78400043f0 con 0x7f784c0d3160 2024-09-12T01:34:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.152+0000 7f78524a4640 1 -- 172.21.15.16:0/301298166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784c155cd0 con 0x7f784c0d3160 2024-09-12T01:34:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.152+0000 7f7848ff9640 1 -- 172.21.15.16:0/301298166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840007860 con 0x7f784c0d3160 2024-09-12T01:34:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.152+0000 7f78524a4640 1 -- 172.21.15.16:0/301298166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784c156290 con 0x7f784c0d3160 2024-09-12T01:34:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.153+0000 7f78524a4640 1 -- 172.21.15.16:0/301298166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784c0d4b30 con 0x7f784c0d3160 2024-09-12T01:34:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.153+0000 7f7848ff9640 1 -- 172.21.15.16:0/301298166 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7840007c10 con 0x7f784c0d3160 2024-09-12T01:34:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.154+0000 7f7848ff9640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f783806be30 0x7f783806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.154+0000 7f7848ff9640 1 -- 172.21.15.16:0/301298166 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f784008b280 con 0x7f784c0d3160 2024-09-12T01:34:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.154+0000 7f784bfff640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f783806be30 0x7f783806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.154+0000 7f784bfff640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f783806be30 0x7f783806e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f783c006fd0 tx=0x7f783c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:09.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.158+0000 7f7848ff9640 1 -- 172.21.15.16:0/301298166 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7840015020 con 0x7f784c0d3160 2024-09-12T01:34:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:09.375+0000 7f78524a4640 1 -- 172.21.15.16:0/301298166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f784c0d0c10 con 0x7f784c0d3160 2024-09-12T01:34:10.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.019+0000 7f7848ff9640 1 -- 172.21.15.16:0/301298166 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f78400593c0 con 0x7f784c0d3160 2024-09-12T01:34:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.022+0000 7f78524a4640 1 -- 172.21.15.16:0/301298166 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f783806be30 msgr2=0x7f783806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.022+0000 7f78524a4640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f783806be30 0x7f783806e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f783c006fd0 tx=0x7f783c008040 comp rx=0 tx=0).stop 2024-09-12T01:34:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.022+0000 7f78524a4640 1 -- 172.21.15.16:0/301298166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784c0d3160 msgr2=0x7f784c155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.022+0000 7f78524a4640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784c0d3160 0x7f784c155250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f784000c7f0 tx=0x7f784000ccb0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.022+0000 7f78524a4640 1 -- 172.21.15.16:0/301298166 shutdown_connections 2024-09-12T01:34:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.023+0000 7f78524a4640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f783806be30 0x7f783806e2f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.023+0000 7f78524a4640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f784c15e6a0 0x7f784c0c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.023+0000 7f78524a4640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784c0d3160 0x7f784c155250 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.023+0000 7f78524a4640 1 --2- 172.21.15.16:0/301298166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f784c0d0e30 0x7f784c157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.023+0000 7f78524a4640 1 -- 172.21.15.16:0/301298166 >> 172.21.15.16:0/301298166 conn(0x7f784c09d730 msgr2=0x7f784c0b9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.023+0000 7f78524a4640 1 -- 172.21.15.16:0/301298166 shutdown_connections 2024-09-12T01:34:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.023+0000 7f78524a4640 1 -- 172.21.15.16:0/301298166 wait complete. 2024-09-12T01:34:10.036 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T01:34:10.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:34:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.146+0000 7fa7f172a640 1 Processor -- start 2024-09-12T01:34:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.146+0000 7fa7f172a640 1 -- start start 2024-09-12T01:34:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.146+0000 7fa7f172a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa7ec0d0350 0x7fa7ec06cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.146+0000 7fa7f172a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa7ec0b1930 0x7fa7ec0b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.146+0000 7fa7f172a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7ec15e7f0 con 0x7fa7ec09aa30 2024-09-12T01:34:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.146+0000 7fa7f172a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa7ec0d1a50 con 0x7fa7ec0b1930 2024-09-12T01:34:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.146+0000 7fa7f172a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fa7ec0d09b0 con 0x7fa7ec0d0350 2024-09-12T01:34:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.147+0000 7fa7eaffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa7ec09aa30 0x7fa7ec09ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55086/0 (socket says 172.21.15.16:55086) 2024-09-12T01:34:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.147+0000 7fa7eaffd640 1 -- 172.21.15.16:0/1857837188 learned_addr learned my addr 172.21.15.16:0/1857837188 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.147+0000 7fa7eb7fe640 1 --2- 172.21.15.16:0/1857837188 >> v2:172.21.15.16:3300/0 conn(0x7fa7ec0b1930 0x7fa7ec0b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.147+0000 7fa7ea7fc640 1 --2- 172.21.15.16:0/1857837188 >> v2:172.21.15.33:3300/0 conn(0x7fa7ec0d0350 0x7fa7ec06cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.147+0000 7fa7e9ffb640 1 -- 172.21.15.16:0/1857837188 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1892475295 0 0) 0x7fa7ec15e7f0 con 0x7fa7ec09aa30 2024-09-12T01:34:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.147+0000 7fa7e9ffb640 1 -- 172.21.15.16:0/1857837188 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7d8003520 con 0x7fa7ec09aa30 2024-09-12T01:34:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.147+0000 7fa7eb7fe640 1 -- 172.21.15.16:0/1857837188 >> v2:172.21.15.33:3300/0 conn(0x7fa7ec0d0350 msgr2=0x7fa7ec06cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.147+0000 7fa7eb7fe640 1 --2- 172.21.15.16:0/1857837188 >> v2:172.21.15.33:3300/0 conn(0x7fa7ec0d0350 0x7fa7ec06cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.147+0000 7fa7eb7fe640 1 -- 172.21.15.16:0/1857837188 >> v1:172.21.15.16:6789/0 conn(0x7fa7ec09aa30 legacy=0x7fa7ec09ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.147+0000 7fa7eb7fe640 1 -- 172.21.15.16:0/1857837188 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ec0d0c90 con 0x7fa7ec0b1930 2024-09-12T01:34:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.148+0000 7fa7eb7fe640 1 --2- 172.21.15.16:0/1857837188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ec0b1930 0x7fa7ec0b1d40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0730b0 tx=0x7fa7e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ab2d8525c7d897 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.148+0000 7fa7e9ffb640 1 -- 172.21.15.16:0/1857837188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4037070 con 0x7fa7ec0b1930 2024-09-12T01:34:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.148+0000 7fa7e9ffb640 1 -- 172.21.15.16:0/1857837188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e4032440 con 0x7fa7ec0b1930 2024-09-12T01:34:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.148+0000 7fa7e9ffb640 1 -- 172.21.15.16:0/1857837188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40327f0 con 0x7fa7ec0b1930 2024-09-12T01:34:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.148+0000 7fa7f172a640 1 -- 172.21.15.16:0/1857837188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ec0b1930 msgr2=0x7fa7ec0b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.148+0000 7fa7f172a640 1 --2- 172.21.15.16:0/1857837188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ec0b1930 0x7fa7ec0b1d40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0730b0 tx=0x7fa7e402a1a0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.148+0000 7fa7f172a640 1 -- 172.21.15.16:0/1857837188 shutdown_connections 2024-09-12T01:34:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.148+0000 7fa7f172a640 1 --2- 172.21.15.16:0/1857837188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ec0b1930 0x7fa7ec0b1d40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.148+0000 7fa7f172a640 1 --2- 172.21.15.16:0/1857837188 >> v2:172.21.15.33:3300/0 conn(0x7fa7ec0d0350 0x7fa7ec06cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.149+0000 7fa7f172a640 1 -- 172.21.15.16:0/1857837188 >> 172.21.15.16:0/1857837188 conn(0x7fa7ec0b3cf0 msgr2=0x7fa7ec0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.149+0000 7fa7f172a640 1 -- 172.21.15.16:0/1857837188 shutdown_connections 2024-09-12T01:34:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.149+0000 7fa7f172a640 1 -- 172.21.15.16:0/1857837188 wait complete. 2024-09-12T01:34:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.150+0000 7fa7f172a640 1 Processor -- start 2024-09-12T01:34:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.150+0000 7fa7f172a640 1 -- start start 2024-09-12T01:34:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7f172a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ec09aa30 0x7fa7ec0c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7f172a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7ec0b1930 0x7fa7ec0c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7f172a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7ec0d0350 0x7fa7ec15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7f172a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec0d1530 con 0x7fa7ec09aa30 2024-09-12T01:34:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7f172a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec0d1250 con 0x7fa7ec0d0350 2024-09-12T01:34:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7f172a640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa7ec06d140 con 0x7fa7ec0b1930 2024-09-12T01:34:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7eaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ec09aa30 0x7fa7ec0c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7eaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ec09aa30 0x7fa7ec0c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41386/0 (socket says 172.21.15.16:41386) 2024-09-12T01:34:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7eaffd640 1 -- 172.21.15.16:0/3765852108 learned_addr learned my addr 172.21.15.16:0/3765852108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7eb7fe640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7ec0d0350 0x7fa7ec15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7ea7fc640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7ec0b1930 0x7fa7ec0c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7eaffd640 1 -- 172.21.15.16:0/3765852108 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7ec0b1930 msgr2=0x7fa7ec0c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7eaffd640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7ec0b1930 0x7fa7ec0c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7eaffd640 1 -- 172.21.15.16:0/3765852108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7ec0d0350 msgr2=0x7fa7ec15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7eaffd640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7ec0d0350 0x7fa7ec15d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7eaffd640 1 -- 172.21.15.16:0/3765852108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ec1e8440 con 0x7fa7ec09aa30 2024-09-12T01:34:10.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7ea7fc640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7ec0b1930 0x7fa7ec0c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:10.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7eaffd640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ec09aa30 0x7fa7ec0c8be0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00cb40 tx=0x7fa7dc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:10.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7e37fe640 1 -- 172.21.15.16:0/3765852108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc017070 con 0x7fa7ec09aa30 2024-09-12T01:34:10.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7e37fe640 1 -- 172.21.15.16:0/3765852108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7dc00aca0 con 0x7fa7ec09aa30 2024-09-12T01:34:10.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.151+0000 7fa7f172a640 1 -- 172.21.15.16:0/3765852108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ec1e8720 con 0x7fa7ec09aa30 2024-09-12T01:34:10.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.152+0000 7fa7e37fe640 1 -- 172.21.15.16:0/3765852108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc012620 con 0x7fa7ec09aa30 2024-09-12T01:34:10.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.152+0000 7fa7f172a640 1 -- 172.21.15.16:0/3765852108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ec1e91e0 con 0x7fa7ec09aa30 2024-09-12T01:34:10.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.153+0000 7fa7f172a640 1 -- 172.21.15.16:0/3765852108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7ec06cc00 con 0x7fa7ec09aa30 2024-09-12T01:34:10.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.153+0000 7fa7e37fe640 1 -- 172.21.15.16:0/3765852108 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa7dc01a070 con 0x7fa7ec09aa30 2024-09-12T01:34:10.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.154+0000 7fa7e37fe640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa7cc06be30 0x7fa7cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.154+0000 7fa7e37fe640 1 -- 172.21.15.16:0/3765852108 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fa7dc08ad20 con 0x7fa7ec09aa30 2024-09-12T01:34:10.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.154+0000 7fa7ea7fc640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa7cc06be30 0x7fa7cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.154+0000 7fa7ea7fc640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa7cc06be30 0x7fa7cc06e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec15e1e0 tx=0x7fa7d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:10.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.158+0000 7fa7e37fe640 1 -- 172.21.15.16:0/3765852108 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa7dc058f80 con 0x7fa7ec09aa30 2024-09-12T01:34:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.371+0000 7fa7f172a640 1 -- 172.21.15.16:0/3765852108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7ec1619d0 con 0x7fa7ec09aa30 2024-09-12T01:34:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.371+0000 7fa7e37fe640 1 -- 172.21.15.16:0/3765852108 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5214 (secure 0 0 0) 0x7fa7dc05d280 con 0x7fa7ec09aa30 2024-09-12T01:34:10.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:10.371 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-12T01:34:10:019140+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/842803487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":842803487},{"type":"v1","addr":"172.21.15.33:6833","nonce":842803487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/70052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":70052721},{"type":"v1","addr":"172.21.15.33:6835","nonce":70052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2095784417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2095784417},{"type":"v1","addr":"172.21.15.16:6835","nonce":2095784417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:34:06.918906+0000","modified":"2024-09-12T01:34:09.376867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4932":{"gid":4932,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/4214019228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4214019228},{"type":"v1","addr":"172.21.15.16:6837","nonce":4214019228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4932,"qdb_cluster":[4932]},"id":2}]} 2024-09-12T01:34:10.372 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-12T01:34:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.373+0000 7fa7f172a640 1 -- 172.21.15.16:0/3765852108 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa7cc06be30 msgr2=0x7fa7cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.373+0000 7fa7f172a640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa7cc06be30 0x7fa7cc06e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec15e1e0 tx=0x7fa7d4008040 comp rx=0 tx=0).stop 2024-09-12T01:34:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.373+0000 7fa7f172a640 1 -- 172.21.15.16:0/3765852108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ec09aa30 msgr2=0x7fa7ec0c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.373+0000 7fa7f172a640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ec09aa30 0x7fa7ec0c8be0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00cb40 tx=0x7fa7dc00a060 comp rx=0 tx=0).stop 2024-09-12T01:34:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.373+0000 7fa7f172a640 1 -- 172.21.15.16:0/3765852108 shutdown_connections 2024-09-12T01:34:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.373+0000 7fa7f172a640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa7cc06be30 0x7fa7cc06e2f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.373+0000 7fa7f172a640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7ec0d0350 0x7fa7ec15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.373+0000 7fa7f172a640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7ec0b1930 0x7fa7ec0c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.373+0000 7fa7f172a640 1 --2- 172.21.15.16:0/3765852108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ec09aa30 0x7fa7ec0c8be0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.373+0000 7fa7f172a640 1 -- 172.21.15.16:0/3765852108 >> 172.21.15.16:0/3765852108 conn(0x7fa7ec0b3cf0 msgr2=0x7fa7ec0af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.374+0000 7fa7f172a640 1 -- 172.21.15.16:0/3765852108 shutdown_connections 2024-09-12T01:34:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.374+0000 7fa7f172a640 1 -- 172.21.15.16:0/3765852108 wait complete. 2024-09-12T01:34:10.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:34:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.487+0000 7f598830a640 1 Processor -- start 2024-09-12T01:34:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.487+0000 7f598830a640 1 -- start start 2024-09-12T01:34:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.488+0000 7f598830a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f59800d04c0 0x7f5980071510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.488+0000 7f598830a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f59801476d0 0x7f59800ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.488+0000 7f598830a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f598015e7c0 con 0x7f59800cf340 2024-09-12T01:34:10.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.488+0000 7f598830a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f5980071b80 con 0x7f59801476d0 2024-09-12T01:34:10.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.488+0000 7f598830a640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f5980071e60 con 0x7f59800d04c0 2024-09-12T01:34:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.488+0000 7f59858a5640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f59800d04c0 0x7f5980071510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.488+0000 7f59858a5640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f59800d04c0 0x7f5980071510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:39224/0 (socket says 172.21.15.16:39224) 2024-09-12T01:34:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.488+0000 7f59858a5640 1 -- 172.21.15.16:0/326555748 learned_addr learned my addr 172.21.15.16:0/326555748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.488+0000 7f59868a7640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f59801476d0 0x7f59800ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.489+0000 7f59858a5640 1 -- 172.21.15.16:0/326555748 >> v2:172.21.15.33:3300/0 conn(0x7f59801476d0 msgr2=0x7f59800ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.489+0000 7f59858a5640 1 --2- 172.21.15.16:0/326555748 >> v2:172.21.15.33:3300/0 conn(0x7f59801476d0 0x7f59800ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.489+0000 7f59858a5640 1 -- 172.21.15.16:0/326555748 >> v1:172.21.15.33:6789/0 conn(0x7f59800cf340 legacy=0x7f59800cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.489+0000 7f59858a5640 1 -- 172.21.15.16:0/326555748 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5980072140 con 0x7f59800d04c0 2024-09-12T01:34:10.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.489+0000 7f59858a5640 1 --2- 172.21.15.16:0/326555748 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f59800d04c0 0x7f5980071510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5970009870 tx=0x7f597002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca0f9702e2cb8f9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:10.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.490+0000 7f59850a4640 1 -- 172.21.15.16:0/326555748 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970037070 con 0x7f59800d04c0 2024-09-12T01:34:10.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.490+0000 7f59850a4640 1 -- 172.21.15.16:0/326555748 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5970032440 con 0x7f59800d04c0 2024-09-12T01:34:10.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.490+0000 7f59850a4640 1 -- 172.21.15.16:0/326555748 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970032810 con 0x7f59800d04c0 2024-09-12T01:34:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.490+0000 7f598830a640 1 -- 172.21.15.16:0/326555748 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f59800d04c0 msgr2=0x7f5980071510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.490+0000 7f598830a640 1 --2- 172.21.15.16:0/326555748 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f59800d04c0 0x7f5980071510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5970009870 tx=0x7f597002a1c0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.490+0000 7f598830a640 1 -- 172.21.15.16:0/326555748 shutdown_connections 2024-09-12T01:34:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.490+0000 7f598830a640 1 --2- 172.21.15.16:0/326555748 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f59800d04c0 0x7f5980071510 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.490+0000 7f598830a640 1 --2- 172.21.15.16:0/326555748 >> v2:172.21.15.33:3300/0 conn(0x7f59801476d0 0x7f59800ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.490+0000 7f598830a640 1 -- 172.21.15.16:0/326555748 >> 172.21.15.16:0/326555748 conn(0x7f59800b3d50 msgr2=0x7f59800b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.490+0000 7f598830a640 1 -- 172.21.15.16:0/326555748 shutdown_connections 2024-09-12T01:34:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.490+0000 7f598830a640 1 -- 172.21.15.16:0/326555748 wait complete. 2024-09-12T01:34:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.491+0000 7f598830a640 1 Processor -- start 2024-09-12T01:34:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.491+0000 7f598830a640 1 -- start start 2024-09-12T01:34:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.492+0000 7f598830a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59800cf340 0x7f59800b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.492+0000 7f598830a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59800d04c0 0x7f59800ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.492+0000 7f598830a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f59801476d0 0x7f5980155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.492+0000 7f598830a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59800cd150 con 0x7f59800d04c0 2024-09-12T01:34:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.492+0000 7f598830a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5980147ee0 con 0x7f59800cf340 2024-09-12T01:34:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.492+0000 7f598830a640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f59800cd2d0 con 0x7f59801476d0 2024-09-12T01:34:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.492+0000 7f59858a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59800d04c0 0x7f59800ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.492+0000 7f59858a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59800d04c0 0x7f59800ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41396/0 (socket says 172.21.15.16:41396) 2024-09-12T01:34:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.492+0000 7f59858a5640 1 -- 172.21.15.16:0/1392746543 learned_addr learned my addr 172.21.15.16:0/1392746543 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.492+0000 7f59868a7640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f59801476d0 0x7f5980155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.492+0000 7f59860a6640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59800cf340 0x7f59800b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.493+0000 7f59858a5640 1 -- 172.21.15.16:0/1392746543 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f59801476d0 msgr2=0x7f5980155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.493+0000 7f59858a5640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f59801476d0 0x7f5980155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.493+0000 7f59858a5640 1 -- 172.21.15.16:0/1392746543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59800cf340 msgr2=0x7f59800b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.493+0000 7f59858a5640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59800cf340 0x7f59800b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.493+0000 7f59858a5640 1 -- 172.21.15.16:0/1392746543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5980155b40 con 0x7f59800d04c0 2024-09-12T01:34:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.493+0000 7f59868a7640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f59801476d0 0x7f5980155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.493+0000 7f59858a5640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59800d04c0 0x7f59800ca1a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5970000c00 tx=0x7f59700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.493+0000 7f5976ffd640 1 -- 172.21.15.16:0/1392746543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970037040 con 0x7f59800d04c0 2024-09-12T01:34:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.493+0000 7f5976ffd640 1 -- 172.21.15.16:0/1392746543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5970042070 con 0x7f59800d04c0 2024-09-12T01:34:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.493+0000 7f5976ffd640 1 -- 172.21.15.16:0/1392746543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970030860 con 0x7f59800d04c0 2024-09-12T01:34:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.493+0000 7f598830a640 1 -- 172.21.15.16:0/1392746543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59800b5fc0 con 0x7f59800d04c0 2024-09-12T01:34:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.493+0000 7f598830a640 1 -- 172.21.15.16:0/1392746543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59800b6580 con 0x7f59800d04c0 2024-09-12T01:34:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.494+0000 7f598830a640 1 -- 172.21.15.16:0/1392746543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5980071510 con 0x7f59800d04c0 2024-09-12T01:34:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.495+0000 7f5976ffd640 1 -- 172.21.15.16:0/1392746543 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f597002ad10 con 0x7f59800d04c0 2024-09-12T01:34:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.495+0000 7f5976ffd640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f596406be60 0x7f596406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.495+0000 7f5976ffd640 1 -- 172.21.15.16:0/1392746543 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f59700ab5a0 con 0x7f59800d04c0 2024-09-12T01:34:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.495+0000 7f59860a6640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f596406be60 0x7f596406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.496+0000 7f59860a6640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f596406be60 0x7f596406e320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f597c001640 tx=0x7f597c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.499+0000 7f5976ffd640 1 -- 172.21.15.16:0/1392746543 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5970079d40 con 0x7f59800d04c0 2024-09-12T01:34:10.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.677+0000 7f598830a640 1 -- 172.21.15.16:0/1392746543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f598015e030 con 0x7f59800d04c0 2024-09-12T01:34:10.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.678+0000 7f5976ffd640 1 -- 172.21.15.16:0/1392746543 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16978 (secure 0 0 0) 0x7f5970079640 con 0x7f59800d04c0 2024-09-12T01:34:10.678 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:10.678 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:34:08.561979+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:34:04.823039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-12T01:34:05.900693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:34:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.680+0000 7f598830a640 1 -- 172.21.15.16:0/1392746543 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f596406be60 msgr2=0x7f596406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.681+0000 7f598830a640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f596406be60 0x7f596406e320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f597c001640 tx=0x7f597c009040 comp rx=0 tx=0).stop 2024-09-12T01:34:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.681+0000 7f598830a640 1 -- 172.21.15.16:0/1392746543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59800d04c0 msgr2=0x7f59800ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.681+0000 7f598830a640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59800d04c0 0x7f59800ca1a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5970000c00 tx=0x7f59700046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.681+0000 7f598830a640 1 -- 172.21.15.16:0/1392746543 shutdown_connections 2024-09-12T01:34:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.681+0000 7f598830a640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f596406be60 0x7f596406e320 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.681+0000 7f598830a640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f59801476d0 0x7f5980155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.681+0000 7f598830a640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59800d04c0 0x7f59800ca1a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.681+0000 7f598830a640 1 --2- 172.21.15.16:0/1392746543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59800cf340 0x7f59800b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.681+0000 7f598830a640 1 -- 172.21.15.16:0/1392746543 >> 172.21.15.16:0/1392746543 conn(0x7f59800b3d50 msgr2=0x7f59800cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.681+0000 7f598830a640 1 -- 172.21.15.16:0/1392746543 shutdown_connections 2024-09-12T01:34:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.681+0000 7f598830a640 1 -- 172.21.15.16:0/1392746543 wait complete. 2024-09-12T01:34:10.693 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T01:34:10.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:34:10.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.797+0000 7f2fef3fd640 1 Processor -- start 2024-09-12T01:34:10.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.797+0000 7f2fef3fd640 1 -- start start 2024-09-12T01:34:10.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.798+0000 7f2fef3fd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2fe80d17c0 0x7f2fe80b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.798+0000 7f2fef3fd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe815e8b0 con 0x7f2fe80d1350 2024-09-12T01:34:10.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.798+0000 7f2fef3fd640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe80b7510 con 0x7f2fe815e4d0 2024-09-12T01:34:10.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.798+0000 7f2fef3fd640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe80d3bf0 con 0x7f2fe80d17c0 2024-09-12T01:34:10.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.798+0000 7f2fec998640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2fe80d1350 0x7f2fe80b7c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55098/0 (socket says 172.21.15.16:55098) 2024-09-12T01:34:10.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.798+0000 7f2fec998640 1 -- 172.21.15.16:0/230210399 learned_addr learned my addr 172.21.15.16:0/230210399 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:10.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.798+0000 7f2fed99a640 1 --2- 172.21.15.16:0/230210399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2fe80d17c0 0x7f2fe80b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.798+0000 7f2fdffff640 1 -- 172.21.15.16:0/230210399 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350601175 0 0) 0x7f2fe815e8b0 con 0x7f2fe80d1350 2024-09-12T01:34:10.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.798+0000 7f2fdffff640 1 -- 172.21.15.16:0/230210399 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fd4003520 con 0x7f2fe80d1350 2024-09-12T01:34:10.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.799+0000 7f2fdffff640 1 -- 172.21.15.16:0/230210399 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1474210031 0 0) 0x7f2fe80b7510 con 0x7f2fe815e4d0 2024-09-12T01:34:10.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.799+0000 7f2fdffff640 1 -- 172.21.15.16:0/230210399 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fe815e8b0 con 0x7f2fe815e4d0 2024-09-12T01:34:10.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.799+0000 7f2fdffff640 1 -- 172.21.15.16:0/230210399 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 75003951 0 0) 0x7f2fd4003520 con 0x7f2fe80d1350 2024-09-12T01:34:10.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.799+0000 7f2fed99a640 1 -- 172.21.15.16:0/230210399 >> v1:172.21.15.33:6789/0 conn(0x7f2fe815e4d0 legacy=0x7f2fe80cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.799+0000 7f2fed99a640 1 -- 172.21.15.16:0/230210399 >> v1:172.21.15.16:6789/0 conn(0x7f2fe80d1350 legacy=0x7f2fe80b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.799+0000 7f2fed99a640 1 -- 172.21.15.16:0/230210399 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe80b77b0 con 0x7f2fe80d17c0 2024-09-12T01:34:10.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.800+0000 7f2fed99a640 1 --2- 172.21.15.16:0/230210399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2fe80d17c0 0x7f2fe80b83a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80b7980 tx=0x7f2fe002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df88e8ebacd7744 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:10.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.800+0000 7f2fdffff640 1 -- 172.21.15.16:0/230210399 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8001690 con 0x7f2fe80d17c0 2024-09-12T01:34:10.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.800+0000 7f2fdffff640 1 -- 172.21.15.16:0/230210399 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe0007540 con 0x7f2fe80d17c0 2024-09-12T01:34:10.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.800+0000 7f2fdffff640 1 -- 172.21.15.16:0/230210399 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00078f0 con 0x7f2fe80d17c0 2024-09-12T01:34:10.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.800+0000 7f2fef3fd640 1 -- 172.21.15.16:0/230210399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2fe80d17c0 msgr2=0x7f2fe80b83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.800+0000 7f2fef3fd640 1 --2- 172.21.15.16:0/230210399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2fe80d17c0 0x7f2fe80b83a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80b7980 tx=0x7f2fe002a660 comp rx=0 tx=0).stop 2024-09-12T01:34:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.800+0000 7f2fef3fd640 1 -- 172.21.15.16:0/230210399 shutdown_connections 2024-09-12T01:34:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.800+0000 7f2fef3fd640 1 --2- 172.21.15.16:0/230210399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2fe80d17c0 0x7f2fe80b83a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.800+0000 7f2fef3fd640 1 -- 172.21.15.16:0/230210399 >> 172.21.15.16:0/230210399 conn(0x7f2fe8073640 msgr2=0x7f2fe8073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.800+0000 7f2fef3fd640 1 -- 172.21.15.16:0/230210399 shutdown_connections 2024-09-12T01:34:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.800+0000 7f2fef3fd640 1 -- 172.21.15.16:0/230210399 wait complete. 2024-09-12T01:34:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.802+0000 7f2fef3fd640 1 Processor -- start 2024-09-12T01:34:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.802+0000 7f2fef3fd640 1 -- start start 2024-09-12T01:34:10.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.802+0000 7f2fef3fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe80d1350 0x7f2fe814d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.802+0000 7f2fef3fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fe80d17c0 0x7f2fe80b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.802+0000 7f2fef3fd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2fe815e4d0 0x7f2fe80b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.802+0000 7f2fef3fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe80d3350 con 0x7f2fe80d1350 2024-09-12T01:34:10.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.802+0000 7f2fef3fd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe80d3100 con 0x7f2fe80d17c0 2024-09-12T01:34:10.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.802+0000 7f2fef3fd640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe80d3630 con 0x7f2fe815e4d0 2024-09-12T01:34:10.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.802+0000 7f2fed199640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe80d1350 0x7f2fe814d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.802+0000 7f2fed199640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe80d1350 0x7f2fe814d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41406/0 (socket says 172.21.15.16:41406) 2024-09-12T01:34:10.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.802+0000 7f2fed199640 1 -- 172.21.15.16:0/2524174241 learned_addr learned my addr 172.21.15.16:0/2524174241 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:10.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.803+0000 7f2fed99a640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2fe815e4d0 0x7f2fe80b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.803+0000 7f2fec998640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fe80d17c0 0x7f2fe80b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.803+0000 7f2fed199640 1 -- 172.21.15.16:0/2524174241 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2fe815e4d0 msgr2=0x7f2fe80b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.803+0000 7f2fed199640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2fe815e4d0 0x7f2fe80b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.803+0000 7f2fed199640 1 -- 172.21.15.16:0/2524174241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fe80d17c0 msgr2=0x7f2fe80b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.803+0000 7f2fed199640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fe80d17c0 0x7f2fe80b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.803+0000 7f2fed199640 1 -- 172.21.15.16:0/2524174241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe814dd50 con 0x7f2fe80d1350 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.803+0000 7f2fed199640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe80d1350 0x7f2fe814d810 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c430 tx=0x7f2fe400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.803+0000 7f2fddffb640 1 -- 172.21.15.16:0/2524174241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4017070 con 0x7f2fe80d1350 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.803+0000 7f2fddffb640 1 -- 172.21.15.16:0/2524174241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe40052f0 con 0x7f2fe80d1350 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.803+0000 7f2fef3fd640 1 -- 172.21.15.16:0/2524174241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe814e6b0 con 0x7f2fe80d1350 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.803+0000 7f2fddffb640 1 -- 172.21.15.16:0/2524174241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4003830 con 0x7f2fe80d1350 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.803+0000 7f2fef3fd640 1 -- 172.21.15.16:0/2524174241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe814ec70 con 0x7f2fe80d1350 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.804+0000 7f2fef3fd640 1 -- 172.21.15.16:0/2524174241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe80d2250 con 0x7f2fe80d1350 2024-09-12T01:34:10.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.805+0000 7f2fddffb640 1 -- 172.21.15.16:0/2524174241 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2fe401a070 con 0x7f2fe80d1350 2024-09-12T01:34:10.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.805+0000 7f2fddffb640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2fcc06be30 0x7f2fcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:10.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.805+0000 7f2fddffb640 1 -- 172.21.15.16:0/2524174241 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2fe408a390 con 0x7f2fe80d1350 2024-09-12T01:34:10.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.805+0000 7f2fec998640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2fcc06be30 0x7f2fcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:10.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.806+0000 7f2fec998640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2fcc06be30 0x7f2fcc06e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8001eb0 tx=0x7f2fd80016d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:10.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.810+0000 7f2fddffb640 1 -- 172.21.15.16:0/2524174241 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2fe814dd50 con 0x7f2fe80d1350 2024-09-12T01:34:10.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.981+0000 7f2fef3fd640 1 -- 172.21.15.16:0/2524174241 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2fe806baf0 con 0x7f2fcc06be30 2024-09-12T01:34:10.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.987+0000 7f2fddffb640 1 -- 172.21.15.16:0/2524174241 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243699 (secure 0 0 0) 0x7f2fe806baf0 con 0x7f2fcc06be30 2024-09-12T01:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:10.991 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:34:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.990+0000 7f2fef3fd640 1 -- 172.21.15.16:0/2524174241 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2fcc06be30 msgr2=0x7f2fcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.990+0000 7f2fef3fd640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2fcc06be30 0x7f2fcc06e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8001eb0 tx=0x7f2fd80016d0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.990+0000 7f2fef3fd640 1 -- 172.21.15.16:0/2524174241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe80d1350 msgr2=0x7f2fe814d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.990+0000 7f2fef3fd640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe80d1350 0x7f2fe814d810 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c430 tx=0x7f2fe400c8f0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.990+0000 7f2fef3fd640 1 -- 172.21.15.16:0/2524174241 shutdown_connections 2024-09-12T01:34:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.990+0000 7f2fef3fd640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2fcc06be30 0x7f2fcc06e2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.990+0000 7f2fef3fd640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2fe815e4d0 0x7f2fe80b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.990+0000 7f2fef3fd640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fe80d17c0 0x7f2fe80b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.990+0000 7f2fef3fd640 1 --2- 172.21.15.16:0/2524174241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe80d1350 0x7f2fe814d810 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.990+0000 7f2fef3fd640 1 -- 172.21.15.16:0/2524174241 >> 172.21.15.16:0/2524174241 conn(0x7f2fe8073640 msgr2=0x7f2fe8073a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.990+0000 7f2fef3fd640 1 -- 172.21.15.16:0/2524174241 shutdown_connections 2024-09-12T01:34:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:10.990+0000 7f2fef3fd640 1 -- 172.21.15.16:0/2524174241 wait complete. 2024-09-12T01:34:11.001 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-12T01:34:10.482954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592084,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":173,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":94,"acting":94,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":237376,"kb_used_data":4496,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737344,"statfs":{"total":773094113280,"available":772851040256,"internally_reserved":0,"allocated":4603904,"data_stored":2647351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1716,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.937377"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568397+0000","last_change":"2024-09-12T01:34:07.568217+0000","last_active":"2024-09-12T01:34:08.568397+0000","last_peered":"2024-09-12T01:34:08.568397+0000","last_clean":"2024-09-12T01:34:08.568397+0000","last_became_active":"2024-09-12T01:34:07.567979+0000","last_became_peered":"2024-09-12T01:34:07.567979+0000","last_unstale":"2024-09-12T01:34:08.568397+0000","last_undegraded":"2024-09-12T01:34:08.568397+0000","last_fullsized":"2024-09-12T01:34:08.568397+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:07.721403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567960+0000","last_change":"2024-09-12T01:34:07.566615+0000","last_active":"2024-09-12T01:34:08.567960+0000","last_peered":"2024-09-12T01:34:08.567960+0000","last_clean":"2024-09-12T01:34:08.567960+0000","last_became_active":"2024-09-12T01:34:07.566309+0000","last_became_peered":"2024-09-12T01:34:07.566309+0000","last_unstale":"2024-09-12T01:34:08.567960+0000","last_undegraded":"2024-09-12T01:34:08.567960+0000","last_fullsized":"2024-09-12T01:34:08.567960+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:10.428484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557777+0000","last_change":"2024-09-12T01:34:06.557777+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557777+0000","last_undegraded":"2024-09-12T01:34:06.557777+0000","last_fullsized":"2024-09-12T01:34:06.557777+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.576448+0000","last_change":"2024-09-12T01:34:07.574151+0000","last_active":"2024-09-12T01:34:07.576448+0000","last_peered":"2024-09-12T01:34:07.576448+0000","last_clean":"2024-09-12T01:34:07.576448+0000","last_became_active":"2024-09-12T01:34:07.573810+0000","last_became_peered":"2024-09-12T01:34:07.573810+0000","last_unstale":"2024-09-12T01:34:07.576448+0000","last_undegraded":"2024-09-12T01:34:07.576448+0000","last_fullsized":"2024-09-12T01:34:07.576448+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:56.562905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.577590+0000","last_change":"2024-09-12T01:34:07.572918+0000","last_active":"2024-09-12T01:34:07.577590+0000","last_peered":"2024-09-12T01:34:07.577590+0000","last_clean":"2024-09-12T01:34:07.577590+0000","last_became_active":"2024-09-12T01:34:07.572649+0000","last_became_peered":"2024-09-12T01:34:07.572649+0000","last_unstale":"2024-09-12T01:34:07.577590+0000","last_undegraded":"2024-09-12T01:34:07.577590+0000","last_fullsized":"2024-09-12T01:34:07.577590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:37.323845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.580009+0000","last_change":"2024-09-12T01:34:07.571121+0000","last_active":"2024-09-12T01:34:07.580009+0000","last_peered":"2024-09-12T01:34:07.580009+0000","last_clean":"2024-09-12T01:34:07.580009+0000","last_became_active":"2024-09-12T01:34:07.570676+0000","last_became_peered":"2024-09-12T01:34:07.570676+0000","last_unstale":"2024-09-12T01:34:07.580009+0000","last_undegraded":"2024-09-12T01:34:07.580009+0000","last_fullsized":"2024-09-12T01:34:07.580009+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:19.574669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567753+0000","last_change":"2024-09-12T01:34:07.581851+0000","last_active":"2024-09-12T01:34:08.567753+0000","last_peered":"2024-09-12T01:34:08.567753+0000","last_clean":"2024-09-12T01:34:08.567753+0000","last_became_active":"2024-09-12T01:34:07.581562+0000","last_became_peered":"2024-09-12T01:34:07.581562+0000","last_unstale":"2024-09-12T01:34:08.567753+0000","last_undegraded":"2024-09-12T01:34:08.567753+0000","last_fullsized":"2024-09-12T01:34:08.567753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.266349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.559288+0000","last_change":"2024-09-12T01:34:06.559288+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.559288+0000","last_undegraded":"2024-09-12T01:34:06.559288+0000","last_fullsized":"2024-09-12T01:34:06.559288+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.556833+0000","last_change":"2024-09-12T01:34:06.556833+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.556833+0000","last_undegraded":"2024-09-12T01:34:06.556833+0000","last_fullsized":"2024-09-12T01:34:06.556833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.574203+0000","last_change":"2024-09-12T01:34:07.570064+0000","last_active":"2024-09-12T01:34:07.574203+0000","last_peered":"2024-09-12T01:34:07.574203+0000","last_clean":"2024-09-12T01:34:07.574203+0000","last_became_active":"2024-09-12T01:34:07.569772+0000","last_became_peered":"2024-09-12T01:34:07.569772+0000","last_unstale":"2024-09-12T01:34:07.574203+0000","last_undegraded":"2024-09-12T01:34:07.574203+0000","last_fullsized":"2024-09-12T01:34:07.574203+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:21:38.197322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.574111+0000","last_change":"2024-09-12T01:34:07.570352+0000","last_active":"2024-09-12T01:34:07.574111+0000","last_peered":"2024-09-12T01:34:07.574111+0000","last_clean":"2024-09-12T01:34:07.574111+0000","last_became_active":"2024-09-12T01:34:07.569630+0000","last_became_peered":"2024-09-12T01:34:07.569630+0000","last_unstale":"2024-09-12T01:34:07.574111+0000","last_undegraded":"2024-09-12T01:34:07.574111+0000","last_fullsized":"2024-09-12T01:34:07.574111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:40.751596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584594+0000","last_change":"2024-09-12T01:34:07.583580+0000","last_active":"2024-09-12T01:34:07.584594+0000","last_peered":"2024-09-12T01:34:07.584594+0000","last_clean":"2024-09-12T01:34:07.584594+0000","last_became_active":"2024-09-12T01:34:07.580853+0000","last_became_peered":"2024-09-12T01:34:07.580853+0000","last_unstale":"2024-09-12T01:34:07.584594+0000","last_undegraded":"2024-09-12T01:34:07.584594+0000","last_fullsized":"2024-09-12T01:34:07.584594+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:21.696603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584661+0000","last_change":"2024-09-12T01:34:07.580355+0000","last_active":"2024-09-12T01:34:07.584661+0000","last_peered":"2024-09-12T01:34:07.584661+0000","last_clean":"2024-09-12T01:34:07.584661+0000","last_became_active":"2024-09-12T01:34:07.580020+0000","last_became_peered":"2024-09-12T01:34:07.580020+0000","last_unstale":"2024-09-12T01:34:07.584661+0000","last_undegraded":"2024-09-12T01:34:07.584661+0000","last_fullsized":"2024-09-12T01:34:07.584661+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:22.777538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584013+0000","last_change":"2024-09-12T01:34:07.581833+0000","last_active":"2024-09-12T01:34:07.584013+0000","last_peered":"2024-09-12T01:34:07.584013+0000","last_clean":"2024-09-12T01:34:07.584013+0000","last_became_active":"2024-09-12T01:34:07.581526+0000","last_became_peered":"2024-09-12T01:34:07.581526+0000","last_unstale":"2024-09-12T01:34:07.584013+0000","last_undegraded":"2024-09-12T01:34:07.584013+0000","last_fullsized":"2024-09-12T01:34:07.584013+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:28.448686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-12T01:34:06.573376+0000","last_change":"2024-09-12T01:34:06.572433+0000","last_active":"2024-09-12T01:34:06.573376+0000","last_peered":"2024-09-12T01:34:06.573376+0000","last_clean":"2024-09-12T01:34:06.573376+0000","last_became_active":"2024-09-12T01:34:06.572147+0000","last_became_peered":"2024-09-12T01:34:06.572147+0000","last_unstale":"2024-09-12T01:34:06.573376+0000","last_undegraded":"2024-09-12T01:34:06.573376+0000","last_fullsized":"2024-09-12T01:34:06.573376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:32.352391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.581274+0000","last_change":"2024-09-12T01:34:06.573185+0000","last_active":"2024-09-12T01:34:07.581274+0000","last_peered":"2024-09-12T01:34:07.581274+0000","last_clean":"2024-09-12T01:34:07.581274+0000","last_became_active":"2024-09-12T01:34:06.572814+0000","last_became_peered":"2024-09-12T01:34:06.572814+0000","last_unstale":"2024-09-12T01:34:07.581274+0000","last_undegraded":"2024-09-12T01:34:07.581274+0000","last_fullsized":"2024-09-12T01:34:07.581274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:35.191411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568251+0000","last_change":"2024-09-12T01:34:07.575202+0000","last_active":"2024-09-12T01:34:08.568251+0000","last_peered":"2024-09-12T01:34:08.568251+0000","last_clean":"2024-09-12T01:34:08.568251+0000","last_became_active":"2024-09-12T01:34:07.574888+0000","last_became_peered":"2024-09-12T01:34:07.574888+0000","last_unstale":"2024-09-12T01:34:08.568251+0000","last_undegraded":"2024-09-12T01:34:08.568251+0000","last_fullsized":"2024-09-12T01:34:08.568251+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:17.171969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.558243+0000","last_change":"2024-09-12T01:34:06.558243+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.558243+0000","last_undegraded":"2024-09-12T01:34:06.558243+0000","last_fullsized":"2024-09-12T01:34:06.558243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562765+0000","last_change":"2024-09-12T01:34:06.571529+0000","last_active":"2024-09-12T01:34:07.562765+0000","last_peered":"2024-09-12T01:34:07.562765+0000","last_clean":"2024-09-12T01:34:07.562765+0000","last_became_active":"2024-09-12T01:34:06.571192+0000","last_became_peered":"2024-09-12T01:34:06.571192+0000","last_unstale":"2024-09-12T01:34:07.562765+0000","last_undegraded":"2024-09-12T01:34:07.562765+0000","last_fullsized":"2024-09-12T01:34:07.562765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:18.183678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568020+0000","last_change":"2024-09-12T01:34:07.572915+0000","last_active":"2024-09-12T01:34:08.568020+0000","last_peered":"2024-09-12T01:34:08.568020+0000","last_clean":"2024-09-12T01:34:08.568020+0000","last_became_active":"2024-09-12T01:34:07.572585+0000","last_became_peered":"2024-09-12T01:34:07.572585+0000","last_unstale":"2024-09-12T01:34:08.568020+0000","last_undegraded":"2024-09-12T01:34:08.568020+0000","last_fullsized":"2024-09-12T01:34:08.568020+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:31.058724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.576425+0000","last_change":"2024-09-12T01:34:07.573978+0000","last_active":"2024-09-12T01:34:07.576425+0000","last_peered":"2024-09-12T01:34:07.576425+0000","last_clean":"2024-09-12T01:34:07.576425+0000","last_became_active":"2024-09-12T01:34:07.573412+0000","last_became_peered":"2024-09-12T01:34:07.573412+0000","last_unstale":"2024-09-12T01:34:07.576425+0000","last_undegraded":"2024-09-12T01:34:07.576425+0000","last_fullsized":"2024-09-12T01:34:07.576425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:05.655767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568445+0000","last_change":"2024-09-12T01:34:06.566440+0000","last_active":"2024-09-12T01:34:08.568445+0000","last_peered":"2024-09-12T01:34:08.568445+0000","last_clean":"2024-09-12T01:34:08.568445+0000","last_became_active":"2024-09-12T01:34:06.566046+0000","last_became_peered":"2024-09-12T01:34:06.566046+0000","last_unstale":"2024-09-12T01:34:08.568445+0000","last_undegraded":"2024-09-12T01:34:08.568445+0000","last_fullsized":"2024-09-12T01:34:08.568445+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:27.443857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557220+0000","last_change":"2024-09-12T01:34:06.557220+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557220+0000","last_undegraded":"2024-09-12T01:34:06.557220+0000","last_fullsized":"2024-09-12T01:34:06.557220+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568357+0000","last_change":"2024-09-12T01:34:06.576115+0000","last_active":"2024-09-12T01:34:08.568357+0000","last_peered":"2024-09-12T01:34:08.568357+0000","last_clean":"2024-09-12T01:34:08.568357+0000","last_became_active":"2024-09-12T01:34:06.575873+0000","last_became_peered":"2024-09-12T01:34:06.575873+0000","last_unstale":"2024-09-12T01:34:08.568357+0000","last_undegraded":"2024-09-12T01:34:08.568357+0000","last_fullsized":"2024-09-12T01:34:08.568357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.691775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557835+0000","last_change":"2024-09-12T01:34:06.557835+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557835+0000","last_undegraded":"2024-09-12T01:34:06.557835+0000","last_fullsized":"2024-09-12T01:34:06.557835+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568240+0000","last_change":"2024-09-12T01:34:06.566516+0000","last_active":"2024-09-12T01:34:08.568240+0000","last_peered":"2024-09-12T01:34:08.568240+0000","last_clean":"2024-09-12T01:34:08.568240+0000","last_became_active":"2024-09-12T01:34:06.566186+0000","last_became_peered":"2024-09-12T01:34:06.566186+0000","last_unstale":"2024-09-12T01:34:08.568240+0000","last_undegraded":"2024-09-12T01:34:08.568240+0000","last_fullsized":"2024-09-12T01:34:08.568240+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:59:30.145684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567848+0000","last_change":"2024-09-12T01:34:07.567627+0000","last_active":"2024-09-12T01:34:08.567848+0000","last_peered":"2024-09-12T01:34:08.567848+0000","last_clean":"2024-09-12T01:34:08.567848+0000","last_became_active":"2024-09-12T01:34:07.567381+0000","last_became_peered":"2024-09-12T01:34:07.567381+0000","last_unstale":"2024-09-12T01:34:08.567848+0000","last_undegraded":"2024-09-12T01:34:08.567848+0000","last_fullsized":"2024-09-12T01:34:08.567848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:30.783361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562642+0000","last_change":"2024-09-12T01:34:06.573250+0000","last_active":"2024-09-12T01:34:07.562642+0000","last_peered":"2024-09-12T01:34:07.562642+0000","last_clean":"2024-09-12T01:34:07.562642+0000","last_became_active":"2024-09-12T01:34:06.572929+0000","last_became_peered":"2024-09-12T01:34:06.572929+0000","last_unstale":"2024-09-12T01:34:07.562642+0000","last_undegraded":"2024-09-12T01:34:07.562642+0000","last_fullsized":"2024-09-12T01:34:07.562642+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:43.363051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568123+0000","last_change":"2024-09-12T01:34:07.566974+0000","last_active":"2024-09-12T01:34:08.568123+0000","last_peered":"2024-09-12T01:34:08.568123+0000","last_clean":"2024-09-12T01:34:08.568123+0000","last_became_active":"2024-09-12T01:34:07.566688+0000","last_became_peered":"2024-09-12T01:34:07.566688+0000","last_unstale":"2024-09-12T01:34:08.568123+0000","last_undegraded":"2024-09-12T01:34:08.568123+0000","last_fullsized":"2024-09-12T01:34:08.568123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:35.009027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568341+0000","last_change":"2024-09-12T01:34:07.576753+0000","last_active":"2024-09-12T01:34:08.568341+0000","last_peered":"2024-09-12T01:34:08.568341+0000","last_clean":"2024-09-12T01:34:08.568341+0000","last_became_active":"2024-09-12T01:34:07.576461+0000","last_became_peered":"2024-09-12T01:34:07.576461+0000","last_unstale":"2024-09-12T01:34:08.568341+0000","last_undegraded":"2024-09-12T01:34:08.568341+0000","last_fullsized":"2024-09-12T01:34:08.568341+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:12.831326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.589168+0000","last_change":"2024-09-12T01:34:06.572508+0000","last_active":"2024-09-12T01:34:07.589168+0000","last_peered":"2024-09-12T01:34:07.589168+0000","last_clean":"2024-09-12T01:34:07.589168+0000","last_became_active":"2024-09-12T01:34:06.572222+0000","last_became_peered":"2024-09-12T01:34:06.572222+0000","last_unstale":"2024-09-12T01:34:07.589168+0000","last_undegraded":"2024-09-12T01:34:07.589168+0000","last_fullsized":"2024-09-12T01:34:07.589168+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:12.959609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.574094+0000","last_change":"2024-09-12T01:34:07.571161+0000","last_active":"2024-09-12T01:34:07.574094+0000","last_peered":"2024-09-12T01:34:07.574094+0000","last_clean":"2024-09-12T01:34:07.574094+0000","last_became_active":"2024-09-12T01:34:07.570891+0000","last_became_peered":"2024-09-12T01:34:07.570891+0000","last_unstale":"2024-09-12T01:34:07.574094+0000","last_undegraded":"2024-09-12T01:34:07.574094+0000","last_fullsized":"2024-09-12T01:34:07.574094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:02.383752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562901+0000","last_change":"2024-09-12T01:34:06.574086+0000","last_active":"2024-09-12T01:34:07.562901+0000","last_peered":"2024-09-12T01:34:07.562901+0000","last_clean":"2024-09-12T01:34:07.562901+0000","last_became_active":"2024-09-12T01:34:06.573682+0000","last_became_peered":"2024-09-12T01:34:06.573682+0000","last_unstale":"2024-09-12T01:34:07.562901+0000","last_undegraded":"2024-09-12T01:34:07.562901+0000","last_fullsized":"2024-09-12T01:34:07.562901+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:38.064258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584434+0000","last_change":"2024-09-12T01:34:06.563081+0000","last_active":"2024-09-12T01:34:07.584434+0000","last_peered":"2024-09-12T01:34:07.584434+0000","last_clean":"2024-09-12T01:34:07.584434+0000","last_became_active":"2024-09-12T01:34:06.562728+0000","last_became_peered":"2024-09-12T01:34:06.562728+0000","last_unstale":"2024-09-12T01:34:07.584434+0000","last_undegraded":"2024-09-12T01:34:07.584434+0000","last_fullsized":"2024-09-12T01:34:07.584434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:56.551827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.558269+0000","last_change":"2024-09-12T01:34:06.558269+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.558269+0000","last_undegraded":"2024-09-12T01:34:06.558269+0000","last_fullsized":"2024-09-12T01:34:06.558269+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.560878+0000","last_change":"2024-09-12T01:34:06.573266+0000","last_active":"2024-09-12T01:34:07.560878+0000","last_peered":"2024-09-12T01:34:07.560878+0000","last_clean":"2024-09-12T01:34:07.560878+0000","last_became_active":"2024-09-12T01:34:06.572679+0000","last_became_peered":"2024-09-12T01:34:06.572679+0000","last_unstale":"2024-09-12T01:34:07.560878+0000","last_undegraded":"2024-09-12T01:34:07.560878+0000","last_fullsized":"2024-09-12T01:34:07.560878+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:50.930775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568150+0000","last_change":"2024-09-12T01:34:07.572416+0000","last_active":"2024-09-12T01:34:08.568150+0000","last_peered":"2024-09-12T01:34:08.568150+0000","last_clean":"2024-09-12T01:34:08.568150+0000","last_became_active":"2024-09-12T01:34:07.572092+0000","last_became_peered":"2024-09-12T01:34:07.572092+0000","last_unstale":"2024-09-12T01:34:08.568150+0000","last_undegraded":"2024-09-12T01:34:08.568150+0000","last_fullsized":"2024-09-12T01:34:08.568150+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:55.078476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.560460+0000","last_change":"2024-09-12T01:34:06.562339+0000","last_active":"2024-09-12T01:34:07.560460+0000","last_peered":"2024-09-12T01:34:07.560460+0000","last_clean":"2024-09-12T01:34:07.560460+0000","last_became_active":"2024-09-12T01:34:06.561990+0000","last_became_peered":"2024-09-12T01:34:06.561990+0000","last_unstale":"2024-09-12T01:34:07.560460+0000","last_undegraded":"2024-09-12T01:34:07.560460+0000","last_fullsized":"2024-09-12T01:34:07.560460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:09.175440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.583991+0000","last_change":"2024-09-12T01:34:07.581576+0000","last_active":"2024-09-12T01:34:07.583991+0000","last_peered":"2024-09-12T01:34:07.583991+0000","last_clean":"2024-09-12T01:34:07.583991+0000","last_became_active":"2024-09-12T01:34:07.580839+0000","last_became_peered":"2024-09-12T01:34:07.580839+0000","last_unstale":"2024-09-12T01:34:07.583991+0000","last_undegraded":"2024-09-12T01:34:07.583991+0000","last_fullsized":"2024-09-12T01:34:07.583991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:32.726522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.561435+0000","last_change":"2024-09-12T01:34:06.577637+0000","last_active":"2024-09-12T01:34:07.561435+0000","last_peered":"2024-09-12T01:34:07.561435+0000","last_clean":"2024-09-12T01:34:07.561435+0000","last_became_active":"2024-09-12T01:34:06.577338+0000","last_became_peered":"2024-09-12T01:34:06.577338+0000","last_unstale":"2024-09-12T01:34:07.561435+0000","last_undegraded":"2024-09-12T01:34:07.561435+0000","last_fullsized":"2024-09-12T01:34:07.561435+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:44.282477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567958+0000","last_change":"2024-09-12T01:34:07.563962+0000","last_active":"2024-09-12T01:34:08.567958+0000","last_peered":"2024-09-12T01:34:08.567958+0000","last_clean":"2024-09-12T01:34:08.567958+0000","last_became_active":"2024-09-12T01:34:07.563649+0000","last_became_peered":"2024-09-12T01:34:07.563649+0000","last_unstale":"2024-09-12T01:34:08.567958+0000","last_undegraded":"2024-09-12T01:34:08.567958+0000","last_fullsized":"2024-09-12T01:34:08.567958+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:36:19.605889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-12T01:34:06.566559+0000","last_change":"2024-09-12T01:34:06.564220+0000","last_active":"2024-09-12T01:34:06.566559+0000","last_peered":"2024-09-12T01:34:06.566559+0000","last_clean":"2024-09-12T01:34:06.566559+0000","last_became_active":"2024-09-12T01:34:06.563912+0000","last_became_peered":"2024-09-12T01:34:06.563912+0000","last_unstale":"2024-09-12T01:34:06.566559+0000","last_undegraded":"2024-09-12T01:34:06.566559+0000","last_fullsized":"2024-09-12T01:34:06.566559+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:26.470501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557400+0000","last_change":"2024-09-12T01:34:06.557400+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557400+0000","last_undegraded":"2024-09-12T01:34:06.557400+0000","last_fullsized":"2024-09-12T01:34:06.557400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-12T01:34:06.569286+0000","last_change":"2024-09-12T01:34:06.567684+0000","last_active":"2024-09-12T01:34:06.569286+0000","last_peered":"2024-09-12T01:34:06.569286+0000","last_clean":"2024-09-12T01:34:06.569286+0000","last_became_active":"2024-09-12T01:34:06.567184+0000","last_became_peered":"2024-09-12T01:34:06.567184+0000","last_unstale":"2024-09-12T01:34:06.569286+0000","last_undegraded":"2024-09-12T01:34:06.569286+0000","last_fullsized":"2024-09-12T01:34:06.569286+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:23.197060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.580895+0000","last_change":"2024-09-12T01:34:07.573865+0000","last_active":"2024-09-12T01:34:07.580895+0000","last_peered":"2024-09-12T01:34:07.580895+0000","last_clean":"2024-09-12T01:34:07.580895+0000","last_became_active":"2024-09-12T01:34:07.573578+0000","last_became_peered":"2024-09-12T01:34:07.573578+0000","last_unstale":"2024-09-12T01:34:07.580895+0000","last_undegraded":"2024-09-12T01:34:07.580895+0000","last_fullsized":"2024-09-12T01:34:07.580895+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:03.055098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567833+0000","last_change":"2024-09-12T01:34:06.566830+0000","last_active":"2024-09-12T01:34:08.567833+0000","last_peered":"2024-09-12T01:34:08.567833+0000","last_clean":"2024-09-12T01:34:08.567833+0000","last_became_active":"2024-09-12T01:34:06.566321+0000","last_became_peered":"2024-09-12T01:34:06.566321+0000","last_unstale":"2024-09-12T01:34:08.567833+0000","last_undegraded":"2024-09-12T01:34:08.567833+0000","last_fullsized":"2024-09-12T01:34:08.567833+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:31.718104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-12T01:34:02.259557+0000","last_change":"2024-09-12T01:33:01.096542+0000","last_active":"2024-09-12T01:34:02.259557+0000","last_peered":"2024-09-12T01:34:02.259557+0000","last_clean":"2024-09-12T01:34:02.259557+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:34:02.259557+0000","last_undegraded":"2024-09-12T01:34:02.259557+0000","last_fullsized":"2024-09-12T01:34:02.259557+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_clean_scrub_stamp":"2024-09-12T01:32:58.512553+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:48:58.741813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":60,"acting":60,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1716,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":32,"acting":32,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":999424,"data_stored":763507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":173139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":173139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":173139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":471040,"data_stored":201520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":446464,"data_stored":193272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":446464,"data_stored":193252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1024000,"data_stored":776383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:34:11.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:34:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.145+0000 7feba3047640 1 Processor -- start 2024-09-12T01:34:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.145+0000 7feba3047640 1 -- start start 2024-09-12T01:34:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feba3047640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb9c0d04b0 0x7feb9c06e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feba3047640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb9c070be0 con 0x7feb9c15f5b0 2024-09-12T01:34:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feba3047640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb9c15fa90 con 0x7feb9c158cb0 2024-09-12T01:34:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feba3047640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7feb9c1621c0 con 0x7feb9c0d04b0 2024-09-12T01:34:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feba15e4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7feb9c15f5b0 0x7feb9c071540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55104/0 (socket says 172.21.15.16:55104) 2024-09-12T01:34:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feba15e4640 1 -- 172.21.15.16:0/3479012667 learned_addr learned my addr 172.21.15.16:0/3479012667 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feb9bfff640 1 --2- 172.21.15.16:0/3479012667 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb9c0d04b0 0x7feb9c06e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feb9b7fe640 1 -- 172.21.15.16:0/3479012667 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1165533129 0 0) 0x7feb9c070be0 con 0x7feb9c15f5b0 2024-09-12T01:34:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feb9b7fe640 1 -- 172.21.15.16:0/3479012667 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb8c003520 con 0x7feb9c15f5b0 2024-09-12T01:34:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feb9b7fe640 1 -- 172.21.15.16:0/3479012667 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 918878518 0 0) 0x7feb9c15fa90 con 0x7feb9c158cb0 2024-09-12T01:34:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feb9b7fe640 1 -- 172.21.15.16:0/3479012667 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb9c070be0 con 0x7feb9c158cb0 2024-09-12T01:34:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feb9bfff640 1 -- 172.21.15.16:0/3479012667 >> v1:172.21.15.33:6789/0 conn(0x7feb9c158cb0 legacy=0x7feb9c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feb9bfff640 1 -- 172.21.15.16:0/3479012667 >> v1:172.21.15.16:6789/0 conn(0x7feb9c15f5b0 legacy=0x7feb9c071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.146+0000 7feb9bfff640 1 -- 172.21.15.16:0/3479012667 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c1624a0 con 0x7feb9c0d04b0 2024-09-12T01:34:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.147+0000 7feb9bfff640 1 --2- 172.21.15.16:0/3479012667 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb9c0d04b0 0x7feb9c06e780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feb880097d0 tx=0x7feb8802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26cfcb178d337bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.147+0000 7feb9b7fe640 1 -- 172.21.15.16:0/3479012667 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88037070 con 0x7feb9c0d04b0 2024-09-12T01:34:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.147+0000 7feb9b7fe640 1 -- 172.21.15.16:0/3479012667 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb88032440 con 0x7feb9c0d04b0 2024-09-12T01:34:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.147+0000 7feb9b7fe640 1 -- 172.21.15.16:0/3479012667 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88032810 con 0x7feb9c0d04b0 2024-09-12T01:34:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.148+0000 7feba3047640 1 -- 172.21.15.16:0/3479012667 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb9c0d04b0 msgr2=0x7feb9c06e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.148+0000 7feba3047640 1 --2- 172.21.15.16:0/3479012667 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb9c0d04b0 0x7feb9c06e780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feb880097d0 tx=0x7feb8802a210 comp rx=0 tx=0).stop 2024-09-12T01:34:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.148+0000 7feba3047640 1 -- 172.21.15.16:0/3479012667 shutdown_connections 2024-09-12T01:34:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.148+0000 7feba3047640 1 --2- 172.21.15.16:0/3479012667 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb9c0d04b0 0x7feb9c06e780 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.148+0000 7feba3047640 1 -- 172.21.15.16:0/3479012667 >> 172.21.15.16:0/3479012667 conn(0x7feb9c06fdc0 msgr2=0x7feb9c0701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.148+0000 7feba3047640 1 -- 172.21.15.16:0/3479012667 shutdown_connections 2024-09-12T01:34:11.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.148+0000 7feba3047640 1 -- 172.21.15.16:0/3479012667 wait complete. 2024-09-12T01:34:11.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.149+0000 7feba3047640 1 Processor -- start 2024-09-12T01:34:11.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.149+0000 7feba3047640 1 -- start start 2024-09-12T01:34:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.150+0000 7feba3047640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb9c0d04b0 0x7feb9c0b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.150+0000 7feba3047640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb9c158cb0 0x7feb9c0cf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.150+0000 7feba3047640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb9c15f5b0 0x7feb9c154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.150+0000 7feba3047640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c162f80 con 0x7feb9c158cb0 2024-09-12T01:34:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.150+0000 7feba3047640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c162a60 con 0x7feb9c15f5b0 2024-09-12T01:34:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.150+0000 7feba3047640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7feb9c071c50 con 0x7feb9c0d04b0 2024-09-12T01:34:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.150+0000 7feb9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb9c158cb0 0x7feb9c0cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.150+0000 7feb9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb9c158cb0 0x7feb9c0cf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41422/0 (socket says 172.21.15.16:41422) 2024-09-12T01:34:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.150+0000 7feb9bfff640 1 -- 172.21.15.16:0/3844344389 learned_addr learned my addr 172.21.15.16:0/3844344389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.150+0000 7feba15e4640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb9c15f5b0 0x7feb9c154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.150+0000 7feba0de3640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb9c0d04b0 0x7feb9c0b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.151+0000 7feb9bfff640 1 -- 172.21.15.16:0/3844344389 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb9c0d04b0 msgr2=0x7feb9c0b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.151+0000 7feb9bfff640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb9c0d04b0 0x7feb9c0b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.151+0000 7feb9bfff640 1 -- 172.21.15.16:0/3844344389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb9c15f5b0 msgr2=0x7feb9c154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.151+0000 7feb9bfff640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb9c15f5b0 0x7feb9c154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.151+0000 7feb9bfff640 1 -- 172.21.15.16:0/3844344389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c0b7a80 con 0x7feb9c158cb0 2024-09-12T01:34:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.151+0000 7feb9bfff640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb9c158cb0 0x7feb9c0cf230 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feb880097d0 tx=0x7feb880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.151+0000 7feb997fa640 1 -- 172.21.15.16:0/3844344389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88041020 con 0x7feb9c158cb0 2024-09-12T01:34:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.151+0000 7feb997fa640 1 -- 172.21.15.16:0/3844344389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb8803d6e0 con 0x7feb9c158cb0 2024-09-12T01:34:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.151+0000 7feba3047640 1 -- 172.21.15.16:0/3844344389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9c0b7d60 con 0x7feb9c158cb0 2024-09-12T01:34:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.151+0000 7feb997fa640 1 -- 172.21.15.16:0/3844344389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88037040 con 0x7feb9c158cb0 2024-09-12T01:34:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.151+0000 7feba3047640 1 -- 172.21.15.16:0/3844344389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9c157b50 con 0x7feb9c158cb0 2024-09-12T01:34:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.152+0000 7feba3047640 1 -- 172.21.15.16:0/3844344389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9c069dd0 con 0x7feb9c158cb0 2024-09-12T01:34:11.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.153+0000 7feb997fa640 1 -- 172.21.15.16:0/3844344389 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb88044020 con 0x7feb9c158cb0 2024-09-12T01:34:11.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.153+0000 7feb997fa640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feb8006be60 0x7feb8006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.153+0000 7feba0de3640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feb8006be60 0x7feb8006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.153+0000 7feb997fa640 1 -- 172.21.15.16:0/3844344389 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7feb880ab540 con 0x7feb9c158cb0 2024-09-12T01:34:11.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.154+0000 7feba0de3640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feb8006be60 0x7feb8006e320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feb900011e0 tx=0x7feb90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:11.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.158+0000 7feb997fa640 1 -- 172.21.15.16:0/3844344389 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feb9c069dd0 con 0x7feb9c158cb0 2024-09-12T01:34:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.332+0000 7feba3047640 1 -- 172.21.15.16:0/3844344389 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feb9c06e780 con 0x7feb8006be60 2024-09-12T01:34:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.338+0000 7feb997fa640 1 -- 172.21.15.16:0/3844344389 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243699 (secure 0 0 0) 0x7feb9c06e780 con 0x7feb8006be60 2024-09-12T01:34:11.341 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:34:11.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.341+0000 7feba3047640 1 -- 172.21.15.16:0/3844344389 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feb8006be60 msgr2=0x7feb8006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.341+0000 7feba3047640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feb8006be60 0x7feb8006e320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feb900011e0 tx=0x7feb90009040 comp rx=0 tx=0).stop 2024-09-12T01:34:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.341+0000 7feba3047640 1 -- 172.21.15.16:0/3844344389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb9c158cb0 msgr2=0x7feb9c0cf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.341+0000 7feba3047640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb9c158cb0 0x7feb9c0cf230 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feb880097d0 tx=0x7feb880046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.341+0000 7feba3047640 1 -- 172.21.15.16:0/3844344389 shutdown_connections 2024-09-12T01:34:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.341+0000 7feba3047640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feb8006be60 0x7feb8006e320 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.341+0000 7feba3047640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb9c15f5b0 0x7feb9c154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.341+0000 7feba3047640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb9c158cb0 0x7feb9c0cf230 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.341+0000 7feba3047640 1 --2- 172.21.15.16:0/3844344389 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb9c0d04b0 0x7feb9c0b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.341+0000 7feba3047640 1 -- 172.21.15.16:0/3844344389 >> 172.21.15.16:0/3844344389 conn(0x7feb9c06fdc0 msgr2=0x7feb9c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.341+0000 7feba3047640 1 -- 172.21.15.16:0/3844344389 shutdown_connections 2024-09-12T01:34:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.341+0000 7feba3047640 1 -- 172.21.15.16:0/3844344389 wait complete. 2024-09-12T01:34:11.351 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-12T01:34:10.482954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592084,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":173,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":94,"acting":94,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":237376,"kb_used_data":4496,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737344,"statfs":{"total":773094113280,"available":772851040256,"internally_reserved":0,"allocated":4603904,"data_stored":2647351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1716,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.937377"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568397+0000","last_change":"2024-09-12T01:34:07.568217+0000","last_active":"2024-09-12T01:34:08.568397+0000","last_peered":"2024-09-12T01:34:08.568397+0000","last_clean":"2024-09-12T01:34:08.568397+0000","last_became_active":"2024-09-12T01:34:07.567979+0000","last_became_peered":"2024-09-12T01:34:07.567979+0000","last_unstale":"2024-09-12T01:34:08.568397+0000","last_undegraded":"2024-09-12T01:34:08.568397+0000","last_fullsized":"2024-09-12T01:34:08.568397+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:07.721403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567960+0000","last_change":"2024-09-12T01:34:07.566615+0000","last_active":"2024-09-12T01:34:08.567960+0000","last_peered":"2024-09-12T01:34:08.567960+0000","last_clean":"2024-09-12T01:34:08.567960+0000","last_became_active":"2024-09-12T01:34:07.566309+0000","last_became_peered":"2024-09-12T01:34:07.566309+0000","last_unstale":"2024-09-12T01:34:08.567960+0000","last_undegraded":"2024-09-12T01:34:08.567960+0000","last_fullsized":"2024-09-12T01:34:08.567960+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:10.428484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557777+0000","last_change":"2024-09-12T01:34:06.557777+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557777+0000","last_undegraded":"2024-09-12T01:34:06.557777+0000","last_fullsized":"2024-09-12T01:34:06.557777+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.576448+0000","last_change":"2024-09-12T01:34:07.574151+0000","last_active":"2024-09-12T01:34:07.576448+0000","last_peered":"2024-09-12T01:34:07.576448+0000","last_clean":"2024-09-12T01:34:07.576448+0000","last_became_active":"2024-09-12T01:34:07.573810+0000","last_became_peered":"2024-09-12T01:34:07.573810+0000","last_unstale":"2024-09-12T01:34:07.576448+0000","last_undegraded":"2024-09-12T01:34:07.576448+0000","last_fullsized":"2024-09-12T01:34:07.576448+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:56.562905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.577590+0000","last_change":"2024-09-12T01:34:07.572918+0000","last_active":"2024-09-12T01:34:07.577590+0000","last_peered":"2024-09-12T01:34:07.577590+0000","last_clean":"2024-09-12T01:34:07.577590+0000","last_became_active":"2024-09-12T01:34:07.572649+0000","last_became_peered":"2024-09-12T01:34:07.572649+0000","last_unstale":"2024-09-12T01:34:07.577590+0000","last_undegraded":"2024-09-12T01:34:07.577590+0000","last_fullsized":"2024-09-12T01:34:07.577590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:37.323845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.580009+0000","last_change":"2024-09-12T01:34:07.571121+0000","last_active":"2024-09-12T01:34:07.580009+0000","last_peered":"2024-09-12T01:34:07.580009+0000","last_clean":"2024-09-12T01:34:07.580009+0000","last_became_active":"2024-09-12T01:34:07.570676+0000","last_became_peered":"2024-09-12T01:34:07.570676+0000","last_unstale":"2024-09-12T01:34:07.580009+0000","last_undegraded":"2024-09-12T01:34:07.580009+0000","last_fullsized":"2024-09-12T01:34:07.580009+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:19.574669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567753+0000","last_change":"2024-09-12T01:34:07.581851+0000","last_active":"2024-09-12T01:34:08.567753+0000","last_peered":"2024-09-12T01:34:08.567753+0000","last_clean":"2024-09-12T01:34:08.567753+0000","last_became_active":"2024-09-12T01:34:07.581562+0000","last_became_peered":"2024-09-12T01:34:07.581562+0000","last_unstale":"2024-09-12T01:34:08.567753+0000","last_undegraded":"2024-09-12T01:34:08.567753+0000","last_fullsized":"2024-09-12T01:34:08.567753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.266349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.559288+0000","last_change":"2024-09-12T01:34:06.559288+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.559288+0000","last_undegraded":"2024-09-12T01:34:06.559288+0000","last_fullsized":"2024-09-12T01:34:06.559288+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.556833+0000","last_change":"2024-09-12T01:34:06.556833+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.556833+0000","last_undegraded":"2024-09-12T01:34:06.556833+0000","last_fullsized":"2024-09-12T01:34:06.556833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.574203+0000","last_change":"2024-09-12T01:34:07.570064+0000","last_active":"2024-09-12T01:34:07.574203+0000","last_peered":"2024-09-12T01:34:07.574203+0000","last_clean":"2024-09-12T01:34:07.574203+0000","last_became_active":"2024-09-12T01:34:07.569772+0000","last_became_peered":"2024-09-12T01:34:07.569772+0000","last_unstale":"2024-09-12T01:34:07.574203+0000","last_undegraded":"2024-09-12T01:34:07.574203+0000","last_fullsized":"2024-09-12T01:34:07.574203+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:21:38.197322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.574111+0000","last_change":"2024-09-12T01:34:07.570352+0000","last_active":"2024-09-12T01:34:07.574111+0000","last_peered":"2024-09-12T01:34:07.574111+0000","last_clean":"2024-09-12T01:34:07.574111+0000","last_became_active":"2024-09-12T01:34:07.569630+0000","last_became_peered":"2024-09-12T01:34:07.569630+0000","last_unstale":"2024-09-12T01:34:07.574111+0000","last_undegraded":"2024-09-12T01:34:07.574111+0000","last_fullsized":"2024-09-12T01:34:07.574111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:40.751596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584594+0000","last_change":"2024-09-12T01:34:07.583580+0000","last_active":"2024-09-12T01:34:07.584594+0000","last_peered":"2024-09-12T01:34:07.584594+0000","last_clean":"2024-09-12T01:34:07.584594+0000","last_became_active":"2024-09-12T01:34:07.580853+0000","last_became_peered":"2024-09-12T01:34:07.580853+0000","last_unstale":"2024-09-12T01:34:07.584594+0000","last_undegraded":"2024-09-12T01:34:07.584594+0000","last_fullsized":"2024-09-12T01:34:07.584594+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:21.696603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584661+0000","last_change":"2024-09-12T01:34:07.580355+0000","last_active":"2024-09-12T01:34:07.584661+0000","last_peered":"2024-09-12T01:34:07.584661+0000","last_clean":"2024-09-12T01:34:07.584661+0000","last_became_active":"2024-09-12T01:34:07.580020+0000","last_became_peered":"2024-09-12T01:34:07.580020+0000","last_unstale":"2024-09-12T01:34:07.584661+0000","last_undegraded":"2024-09-12T01:34:07.584661+0000","last_fullsized":"2024-09-12T01:34:07.584661+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:22.777538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584013+0000","last_change":"2024-09-12T01:34:07.581833+0000","last_active":"2024-09-12T01:34:07.584013+0000","last_peered":"2024-09-12T01:34:07.584013+0000","last_clean":"2024-09-12T01:34:07.584013+0000","last_became_active":"2024-09-12T01:34:07.581526+0000","last_became_peered":"2024-09-12T01:34:07.581526+0000","last_unstale":"2024-09-12T01:34:07.584013+0000","last_undegraded":"2024-09-12T01:34:07.584013+0000","last_fullsized":"2024-09-12T01:34:07.584013+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:28.448686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-12T01:34:06.573376+0000","last_change":"2024-09-12T01:34:06.572433+0000","last_active":"2024-09-12T01:34:06.573376+0000","last_peered":"2024-09-12T01:34:06.573376+0000","last_clean":"2024-09-12T01:34:06.573376+0000","last_became_active":"2024-09-12T01:34:06.572147+0000","last_became_peered":"2024-09-12T01:34:06.572147+0000","last_unstale":"2024-09-12T01:34:06.573376+0000","last_undegraded":"2024-09-12T01:34:06.573376+0000","last_fullsized":"2024-09-12T01:34:06.573376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:32.352391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.581274+0000","last_change":"2024-09-12T01:34:06.573185+0000","last_active":"2024-09-12T01:34:07.581274+0000","last_peered":"2024-09-12T01:34:07.581274+0000","last_clean":"2024-09-12T01:34:07.581274+0000","last_became_active":"2024-09-12T01:34:06.572814+0000","last_became_peered":"2024-09-12T01:34:06.572814+0000","last_unstale":"2024-09-12T01:34:07.581274+0000","last_undegraded":"2024-09-12T01:34:07.581274+0000","last_fullsized":"2024-09-12T01:34:07.581274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:35.191411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568251+0000","last_change":"2024-09-12T01:34:07.575202+0000","last_active":"2024-09-12T01:34:08.568251+0000","last_peered":"2024-09-12T01:34:08.568251+0000","last_clean":"2024-09-12T01:34:08.568251+0000","last_became_active":"2024-09-12T01:34:07.574888+0000","last_became_peered":"2024-09-12T01:34:07.574888+0000","last_unstale":"2024-09-12T01:34:08.568251+0000","last_undegraded":"2024-09-12T01:34:08.568251+0000","last_fullsized":"2024-09-12T01:34:08.568251+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:17.171969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.558243+0000","last_change":"2024-09-12T01:34:06.558243+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.558243+0000","last_undegraded":"2024-09-12T01:34:06.558243+0000","last_fullsized":"2024-09-12T01:34:06.558243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562765+0000","last_change":"2024-09-12T01:34:06.571529+0000","last_active":"2024-09-12T01:34:07.562765+0000","last_peered":"2024-09-12T01:34:07.562765+0000","last_clean":"2024-09-12T01:34:07.562765+0000","last_became_active":"2024-09-12T01:34:06.571192+0000","last_became_peered":"2024-09-12T01:34:06.571192+0000","last_unstale":"2024-09-12T01:34:07.562765+0000","last_undegraded":"2024-09-12T01:34:07.562765+0000","last_fullsized":"2024-09-12T01:34:07.562765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:18.183678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568020+0000","last_change":"2024-09-12T01:34:07.572915+0000","last_active":"2024-09-12T01:34:08.568020+0000","last_peered":"2024-09-12T01:34:08.568020+0000","last_clean":"2024-09-12T01:34:08.568020+0000","last_became_active":"2024-09-12T01:34:07.572585+0000","last_became_peered":"2024-09-12T01:34:07.572585+0000","last_unstale":"2024-09-12T01:34:08.568020+0000","last_undegraded":"2024-09-12T01:34:08.568020+0000","last_fullsized":"2024-09-12T01:34:08.568020+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:31.058724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.576425+0000","last_change":"2024-09-12T01:34:07.573978+0000","last_active":"2024-09-12T01:34:07.576425+0000","last_peered":"2024-09-12T01:34:07.576425+0000","last_clean":"2024-09-12T01:34:07.576425+0000","last_became_active":"2024-09-12T01:34:07.573412+0000","last_became_peered":"2024-09-12T01:34:07.573412+0000","last_unstale":"2024-09-12T01:34:07.576425+0000","last_undegraded":"2024-09-12T01:34:07.576425+0000","last_fullsized":"2024-09-12T01:34:07.576425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:05.655767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568445+0000","last_change":"2024-09-12T01:34:06.566440+0000","last_active":"2024-09-12T01:34:08.568445+0000","last_peered":"2024-09-12T01:34:08.568445+0000","last_clean":"2024-09-12T01:34:08.568445+0000","last_became_active":"2024-09-12T01:34:06.566046+0000","last_became_peered":"2024-09-12T01:34:06.566046+0000","last_unstale":"2024-09-12T01:34:08.568445+0000","last_undegraded":"2024-09-12T01:34:08.568445+0000","last_fullsized":"2024-09-12T01:34:08.568445+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:27.443857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557220+0000","last_change":"2024-09-12T01:34:06.557220+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557220+0000","last_undegraded":"2024-09-12T01:34:06.557220+0000","last_fullsized":"2024-09-12T01:34:06.557220+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568357+0000","last_change":"2024-09-12T01:34:06.576115+0000","last_active":"2024-09-12T01:34:08.568357+0000","last_peered":"2024-09-12T01:34:08.568357+0000","last_clean":"2024-09-12T01:34:08.568357+0000","last_became_active":"2024-09-12T01:34:06.575873+0000","last_became_peered":"2024-09-12T01:34:06.575873+0000","last_unstale":"2024-09-12T01:34:08.568357+0000","last_undegraded":"2024-09-12T01:34:08.568357+0000","last_fullsized":"2024-09-12T01:34:08.568357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.691775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557835+0000","last_change":"2024-09-12T01:34:06.557835+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557835+0000","last_undegraded":"2024-09-12T01:34:06.557835+0000","last_fullsized":"2024-09-12T01:34:06.557835+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568240+0000","last_change":"2024-09-12T01:34:06.566516+0000","last_active":"2024-09-12T01:34:08.568240+0000","last_peered":"2024-09-12T01:34:08.568240+0000","last_clean":"2024-09-12T01:34:08.568240+0000","last_became_active":"2024-09-12T01:34:06.566186+0000","last_became_peered":"2024-09-12T01:34:06.566186+0000","last_unstale":"2024-09-12T01:34:08.568240+0000","last_undegraded":"2024-09-12T01:34:08.568240+0000","last_fullsized":"2024-09-12T01:34:08.568240+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:59:30.145684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567848+0000","last_change":"2024-09-12T01:34:07.567627+0000","last_active":"2024-09-12T01:34:08.567848+0000","last_peered":"2024-09-12T01:34:08.567848+0000","last_clean":"2024-09-12T01:34:08.567848+0000","last_became_active":"2024-09-12T01:34:07.567381+0000","last_became_peered":"2024-09-12T01:34:07.567381+0000","last_unstale":"2024-09-12T01:34:08.567848+0000","last_undegraded":"2024-09-12T01:34:08.567848+0000","last_fullsized":"2024-09-12T01:34:08.567848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:30.783361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562642+0000","last_change":"2024-09-12T01:34:06.573250+0000","last_active":"2024-09-12T01:34:07.562642+0000","last_peered":"2024-09-12T01:34:07.562642+0000","last_clean":"2024-09-12T01:34:07.562642+0000","last_became_active":"2024-09-12T01:34:06.572929+0000","last_became_peered":"2024-09-12T01:34:06.572929+0000","last_unstale":"2024-09-12T01:34:07.562642+0000","last_undegraded":"2024-09-12T01:34:07.562642+0000","last_fullsized":"2024-09-12T01:34:07.562642+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:43.363051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568123+0000","last_change":"2024-09-12T01:34:07.566974+0000","last_active":"2024-09-12T01:34:08.568123+0000","last_peered":"2024-09-12T01:34:08.568123+0000","last_clean":"2024-09-12T01:34:08.568123+0000","last_became_active":"2024-09-12T01:34:07.566688+0000","last_became_peered":"2024-09-12T01:34:07.566688+0000","last_unstale":"2024-09-12T01:34:08.568123+0000","last_undegraded":"2024-09-12T01:34:08.568123+0000","last_fullsized":"2024-09-12T01:34:08.568123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:35.009027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568341+0000","last_change":"2024-09-12T01:34:07.576753+0000","last_active":"2024-09-12T01:34:08.568341+0000","last_peered":"2024-09-12T01:34:08.568341+0000","last_clean":"2024-09-12T01:34:08.568341+0000","last_became_active":"2024-09-12T01:34:07.576461+0000","last_became_peered":"2024-09-12T01:34:07.576461+0000","last_unstale":"2024-09-12T01:34:08.568341+0000","last_undegraded":"2024-09-12T01:34:08.568341+0000","last_fullsized":"2024-09-12T01:34:08.568341+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:12.831326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.589168+0000","last_change":"2024-09-12T01:34:06.572508+0000","last_active":"2024-09-12T01:34:07.589168+0000","last_peered":"2024-09-12T01:34:07.589168+0000","last_clean":"2024-09-12T01:34:07.589168+0000","last_became_active":"2024-09-12T01:34:06.572222+0000","last_became_peered":"2024-09-12T01:34:06.572222+0000","last_unstale":"2024-09-12T01:34:07.589168+0000","last_undegraded":"2024-09-12T01:34:07.589168+0000","last_fullsized":"2024-09-12T01:34:07.589168+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:12.959609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.574094+0000","last_change":"2024-09-12T01:34:07.571161+0000","last_active":"2024-09-12T01:34:07.574094+0000","last_peered":"2024-09-12T01:34:07.574094+0000","last_clean":"2024-09-12T01:34:07.574094+0000","last_became_active":"2024-09-12T01:34:07.570891+0000","last_became_peered":"2024-09-12T01:34:07.570891+0000","last_unstale":"2024-09-12T01:34:07.574094+0000","last_undegraded":"2024-09-12T01:34:07.574094+0000","last_fullsized":"2024-09-12T01:34:07.574094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:02.383752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562901+0000","last_change":"2024-09-12T01:34:06.574086+0000","last_active":"2024-09-12T01:34:07.562901+0000","last_peered":"2024-09-12T01:34:07.562901+0000","last_clean":"2024-09-12T01:34:07.562901+0000","last_became_active":"2024-09-12T01:34:06.573682+0000","last_became_peered":"2024-09-12T01:34:06.573682+0000","last_unstale":"2024-09-12T01:34:07.562901+0000","last_undegraded":"2024-09-12T01:34:07.562901+0000","last_fullsized":"2024-09-12T01:34:07.562901+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:38.064258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584434+0000","last_change":"2024-09-12T01:34:06.563081+0000","last_active":"2024-09-12T01:34:07.584434+0000","last_peered":"2024-09-12T01:34:07.584434+0000","last_clean":"2024-09-12T01:34:07.584434+0000","last_became_active":"2024-09-12T01:34:06.562728+0000","last_became_peered":"2024-09-12T01:34:06.562728+0000","last_unstale":"2024-09-12T01:34:07.584434+0000","last_undegraded":"2024-09-12T01:34:07.584434+0000","last_fullsized":"2024-09-12T01:34:07.584434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:56.551827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.558269+0000","last_change":"2024-09-12T01:34:06.558269+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.558269+0000","last_undegraded":"2024-09-12T01:34:06.558269+0000","last_fullsized":"2024-09-12T01:34:06.558269+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.560878+0000","last_change":"2024-09-12T01:34:06.573266+0000","last_active":"2024-09-12T01:34:07.560878+0000","last_peered":"2024-09-12T01:34:07.560878+0000","last_clean":"2024-09-12T01:34:07.560878+0000","last_became_active":"2024-09-12T01:34:06.572679+0000","last_became_peered":"2024-09-12T01:34:06.572679+0000","last_unstale":"2024-09-12T01:34:07.560878+0000","last_undegraded":"2024-09-12T01:34:07.560878+0000","last_fullsized":"2024-09-12T01:34:07.560878+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:50.930775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568150+0000","last_change":"2024-09-12T01:34:07.572416+0000","last_active":"2024-09-12T01:34:08.568150+0000","last_peered":"2024-09-12T01:34:08.568150+0000","last_clean":"2024-09-12T01:34:08.568150+0000","last_became_active":"2024-09-12T01:34:07.572092+0000","last_became_peered":"2024-09-12T01:34:07.572092+0000","last_unstale":"2024-09-12T01:34:08.568150+0000","last_undegraded":"2024-09-12T01:34:08.568150+0000","last_fullsized":"2024-09-12T01:34:08.568150+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:55.078476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.560460+0000","last_change":"2024-09-12T01:34:06.562339+0000","last_active":"2024-09-12T01:34:07.560460+0000","last_peered":"2024-09-12T01:34:07.560460+0000","last_clean":"2024-09-12T01:34:07.560460+0000","last_became_active":"2024-09-12T01:34:06.561990+0000","last_became_peered":"2024-09-12T01:34:06.561990+0000","last_unstale":"2024-09-12T01:34:07.560460+0000","last_undegraded":"2024-09-12T01:34:07.560460+0000","last_fullsized":"2024-09-12T01:34:07.560460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:09.175440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.583991+0000","last_change":"2024-09-12T01:34:07.581576+0000","last_active":"2024-09-12T01:34:07.583991+0000","last_peered":"2024-09-12T01:34:07.583991+0000","last_clean":"2024-09-12T01:34:07.583991+0000","last_became_active":"2024-09-12T01:34:07.580839+0000","last_became_peered":"2024-09-12T01:34:07.580839+0000","last_unstale":"2024-09-12T01:34:07.583991+0000","last_undegraded":"2024-09-12T01:34:07.583991+0000","last_fullsized":"2024-09-12T01:34:07.583991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:32.726522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.561435+0000","last_change":"2024-09-12T01:34:06.577637+0000","last_active":"2024-09-12T01:34:07.561435+0000","last_peered":"2024-09-12T01:34:07.561435+0000","last_clean":"2024-09-12T01:34:07.561435+0000","last_became_active":"2024-09-12T01:34:06.577338+0000","last_became_peered":"2024-09-12T01:34:06.577338+0000","last_unstale":"2024-09-12T01:34:07.561435+0000","last_undegraded":"2024-09-12T01:34:07.561435+0000","last_fullsized":"2024-09-12T01:34:07.561435+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:44.282477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567958+0000","last_change":"2024-09-12T01:34:07.563962+0000","last_active":"2024-09-12T01:34:08.567958+0000","last_peered":"2024-09-12T01:34:08.567958+0000","last_clean":"2024-09-12T01:34:08.567958+0000","last_became_active":"2024-09-12T01:34:07.563649+0000","last_became_peered":"2024-09-12T01:34:07.563649+0000","last_unstale":"2024-09-12T01:34:08.567958+0000","last_undegraded":"2024-09-12T01:34:08.567958+0000","last_fullsized":"2024-09-12T01:34:08.567958+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:36:19.605889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-12T01:34:06.566559+0000","last_change":"2024-09-12T01:34:06.564220+0000","last_active":"2024-09-12T01:34:06.566559+0000","last_peered":"2024-09-12T01:34:06.566559+0000","last_clean":"2024-09-12T01:34:06.566559+0000","last_became_active":"2024-09-12T01:34:06.563912+0000","last_became_peered":"2024-09-12T01:34:06.563912+0000","last_unstale":"2024-09-12T01:34:06.566559+0000","last_undegraded":"2024-09-12T01:34:06.566559+0000","last_fullsized":"2024-09-12T01:34:06.566559+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:26.470501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557400+0000","last_change":"2024-09-12T01:34:06.557400+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557400+0000","last_undegraded":"2024-09-12T01:34:06.557400+0000","last_fullsized":"2024-09-12T01:34:06.557400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-12T01:34:06.569286+0000","last_change":"2024-09-12T01:34:06.567684+0000","last_active":"2024-09-12T01:34:06.569286+0000","last_peered":"2024-09-12T01:34:06.569286+0000","last_clean":"2024-09-12T01:34:06.569286+0000","last_became_active":"2024-09-12T01:34:06.567184+0000","last_became_peered":"2024-09-12T01:34:06.567184+0000","last_unstale":"2024-09-12T01:34:06.569286+0000","last_undegraded":"2024-09-12T01:34:06.569286+0000","last_fullsized":"2024-09-12T01:34:06.569286+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:23.197060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.580895+0000","last_change":"2024-09-12T01:34:07.573865+0000","last_active":"2024-09-12T01:34:07.580895+0000","last_peered":"2024-09-12T01:34:07.580895+0000","last_clean":"2024-09-12T01:34:07.580895+0000","last_became_active":"2024-09-12T01:34:07.573578+0000","last_became_peered":"2024-09-12T01:34:07.573578+0000","last_unstale":"2024-09-12T01:34:07.580895+0000","last_undegraded":"2024-09-12T01:34:07.580895+0000","last_fullsized":"2024-09-12T01:34:07.580895+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:03.055098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567833+0000","last_change":"2024-09-12T01:34:06.566830+0000","last_active":"2024-09-12T01:34:08.567833+0000","last_peered":"2024-09-12T01:34:08.567833+0000","last_clean":"2024-09-12T01:34:08.567833+0000","last_became_active":"2024-09-12T01:34:06.566321+0000","last_became_peered":"2024-09-12T01:34:06.566321+0000","last_unstale":"2024-09-12T01:34:08.567833+0000","last_undegraded":"2024-09-12T01:34:08.567833+0000","last_fullsized":"2024-09-12T01:34:08.567833+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:31.718104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-12T01:34:02.259557+0000","last_change":"2024-09-12T01:33:01.096542+0000","last_active":"2024-09-12T01:34:02.259557+0000","last_peered":"2024-09-12T01:34:02.259557+0000","last_clean":"2024-09-12T01:34:02.259557+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:34:02.259557+0000","last_undegraded":"2024-09-12T01:34:02.259557+0000","last_fullsized":"2024-09-12T01:34:02.259557+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_clean_scrub_stamp":"2024-09-12T01:32:58.512553+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:48:58.741813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":60,"acting":60,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1716,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":32,"acting":32,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":999424,"data_stored":763507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":173139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":173139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":173139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":471040,"data_stored":201520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":446464,"data_stored":193272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":446464,"data_stored":193252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1024000,"data_stored":776383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:34:11.355 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:34:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.494+0000 7f03065e4640 1 Processor -- start 2024-09-12T01:34:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.494+0000 7f03065e4640 1 -- start start 2024-09-12T01:34:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.495+0000 7f03065e4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f030015e510 0x7f03000ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.495+0000 7f03065e4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03000d3430 con 0x7f03000cf040 2024-09-12T01:34:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.495+0000 7f03065e4640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03000d36d0 con 0x7f03000d13d0 2024-09-12T01:34:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.495+0000 7f03065e4640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f03000d39b0 con 0x7f030015e510 2024-09-12T01:34:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.495+0000 7f02ff7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f03000cf040 0x7f03000d0cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55108/0 (socket says 172.21.15.16:55108) 2024-09-12T01:34:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.495+0000 7f02ff7fe640 1 -- 172.21.15.16:0/1059648175 learned_addr learned my addr 172.21.15.16:0/1059648175 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02fffff640 1 --2- 172.21.15.16:0/1059648175 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f030015e510 0x7f03000ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3657385566 0 0) 0x7f03000d3430 con 0x7f03000cf040 2024-09-12T01:34:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02e8003520 con 0x7f03000cf040 2024-09-12T01:34:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4186048563 0 0) 0x7f02e8003520 con 0x7f03000cf040 2024-09-12T01:34:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f030015e510 msgr2=0x7f03000ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 --2- 172.21.15.16:0/1059648175 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f030015e510 0x7f03000ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 >> v1:172.21.15.33:6789/0 conn(0x7f03000d13d0 legacy=0x7f03000b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03000d3c90 con 0x7f03000cf040 2024-09-12T01:34:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f02f4001670 con 0x7f03000cf040 2024-09-12T01:34:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 >> v1:172.21.15.16:6789/0 conn(0x7f03000cf040 legacy=0x7f03000d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 --2- 172.21.15.16:0/1059648175 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f030015e510 0x7f02e8003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 --2- 172.21.15.16:0/1059648175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8003f90 0x7f02e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 --2- 172.21.15.16:0/1059648175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02e8004d40 0x7f02e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03000d39b0 con 0x7f02e8003f90 2024-09-12T01:34:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f02e800bb50 con 0x7f02e8004d40 2024-09-12T01:34:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f02e800bd20 con 0x7f030015e510 2024-09-12T01:34:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02ff7fe640 1 --2- 172.21.15.16:0/1059648175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8003f90 0x7f02e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f02fffff640 1 --2- 172.21.15.16:0/1059648175 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f030015e510 0x7f02e8003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.496+0000 7f0304b81640 1 --2- 172.21.15.16:0/1059648175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02e8004d40 0x7f02e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.497+0000 7f02ff7fe640 1 -- 172.21.15.16:0/1059648175 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f030015e510 msgr2=0x7f02e8003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.497+0000 7f02ff7fe640 1 --2- 172.21.15.16:0/1059648175 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f030015e510 0x7f02e8003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.497+0000 7f02ff7fe640 1 -- 172.21.15.16:0/1059648175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02e8004d40 msgr2=0x7f02e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.497+0000 7f02ff7fe640 1 --2- 172.21.15.16:0/1059648175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02e8004d40 0x7f02e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.497+0000 7f02ff7fe640 1 -- 172.21.15.16:0/1059648175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e8003520 con 0x7f02e8003f90 2024-09-12T01:34:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.497+0000 7f02ff7fe640 1 --2- 172.21.15.16:0/1059648175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8003f90 0x7f02e8004800 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f02f4007fd0 tx=0x7f02e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638817c3698155b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.497+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4007d50 con 0x7f02e8003f90 2024-09-12T01:34:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.498+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f40058d0 con 0x7f02e8003f90 2024-09-12T01:34:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.498+0000 7f02feffd640 1 -- 172.21.15.16:0/1059648175 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4005d20 con 0x7f02e8003f90 2024-09-12T01:34:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.498+0000 7f03065e4640 1 -- 172.21.15.16:0/1059648175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8003f90 msgr2=0x7f02e8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.498+0000 7f03065e4640 1 --2- 172.21.15.16:0/1059648175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8003f90 0x7f02e8004800 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f02f4007fd0 tx=0x7f02e8005830 comp rx=0 tx=0).stop 2024-09-12T01:34:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.498+0000 7f0304b81640 1 -- 172.21.15.16:0/1059648175 reap_dead start 2024-09-12T01:34:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.498+0000 7f03065e4640 1 -- 172.21.15.16:0/1059648175 shutdown_connections 2024-09-12T01:34:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.498+0000 7f03065e4640 1 -- 172.21.15.16:0/1059648175 >> 172.21.15.16:0/1059648175 conn(0x7f030006d1a0 msgr2=0x7f0300073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.498+0000 7f03065e4640 1 -- 172.21.15.16:0/1059648175 shutdown_connections 2024-09-12T01:34:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.498+0000 7f03065e4640 1 -- 172.21.15.16:0/1059648175 wait complete. 2024-09-12T01:34:11.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.499+0000 7f03065e4640 1 Processor -- start 2024-09-12T01:34:11.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.499+0000 7f03065e4640 1 -- start start 2024-09-12T01:34:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.500+0000 7f03065e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8003f90 0x7f0300157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.500+0000 7f03065e4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03000ce180 0x7f03001f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.500+0000 7f03065e4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0300155530 0x7f03001559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.500+0000 7f03065e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f030015e8f0 con 0x7f02e8003f90 2024-09-12T01:34:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.500+0000 7f03065e4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03000d22a0 con 0x7f03000ce180 2024-09-12T01:34:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.500+0000 7f03065e4640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f030015ea70 con 0x7f0300155530 2024-09-12T01:34:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.500+0000 7f02fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8003f90 0x7f0300157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.500+0000 7f02fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8003f90 0x7f0300157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41436/0 (socket says 172.21.15.16:41436) 2024-09-12T01:34:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.500+0000 7f02fffff640 1 -- 172.21.15.16:0/371607679 learned_addr learned my addr 172.21.15.16:0/371607679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.501+0000 7f02ff7fe640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03000ce180 0x7f03001f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.501+0000 7f0304b81640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0300155530 0x7f03001559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.501+0000 7f02fffff640 1 -- 172.21.15.16:0/371607679 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0300155530 msgr2=0x7f03001559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.501+0000 7f02fffff640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0300155530 0x7f03001559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.501+0000 7f02fffff640 1 -- 172.21.15.16:0/371607679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03000ce180 msgr2=0x7f03001f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.501+0000 7f02fffff640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03000ce180 0x7f03001f8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.501+0000 7f02fffff640 1 -- 172.21.15.16:0/371607679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03000ce5e0 con 0x7f02e8003f90 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.501+0000 7f02fffff640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8003f90 0x7f0300157400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f02f00119d0 tx=0x7f02f0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.501+0000 7f02fcff9640 1 -- 172.21.15.16:0/371607679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f000fa90 con 0x7f02e8003f90 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.501+0000 7f02fcff9640 1 -- 172.21.15.16:0/371607679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f000fc30 con 0x7f02e8003f90 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.501+0000 7f03065e4640 1 -- 172.21.15.16:0/371607679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0300157a80 con 0x7f02e8003f90 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.501+0000 7f02fcff9640 1 -- 172.21.15.16:0/371607679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0014440 con 0x7f02e8003f90 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.501+0000 7f03065e4640 1 -- 172.21.15.16:0/371607679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0300158040 con 0x7f02e8003f90 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.502+0000 7f03065e4640 1 -- 172.21.15.16:0/371607679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03000ceb00 con 0x7f02e8003f90 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.503+0000 7f02fcff9640 1 -- 172.21.15.16:0/371607679 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02f0012040 con 0x7f02e8003f90 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.503+0000 7f02fcff9640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f02ec06be30 0x7f02ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.503+0000 7f02fcff9640 1 -- 172.21.15.16:0/371607679 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f02f008b5e0 con 0x7f02e8003f90 2024-09-12T01:34:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.503+0000 7f02ff7fe640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f02ec06be30 0x7f02ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.504+0000 7f02ff7fe640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f02ec06be30 0x7f02ec06e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f02f4033000 tx=0x7f02f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.508+0000 7f02fcff9640 1 -- 172.21.15.16:0/371607679 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f02f0019030 con 0x7f02e8003f90 2024-09-12T01:34:11.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.680+0000 7f03065e4640 1 -- 172.21.15.16:0/371607679 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03000cfb50 con 0x7f02ec06be30 2024-09-12T01:34:11.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.686+0000 7f02fcff9640 1 -- 172.21.15.16:0/371607679 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243699 (secure 0 0 0) 0x7f03000cfb50 con 0x7f02ec06be30 2024-09-12T01:34:11.689 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:11.690 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:34:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.689+0000 7f03065e4640 1 -- 172.21.15.16:0/371607679 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f02ec06be30 msgr2=0x7f02ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.689+0000 7f03065e4640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f02ec06be30 0x7f02ec06e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f02f4033000 tx=0x7f02f402e000 comp rx=0 tx=0).stop 2024-09-12T01:34:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.689+0000 7f03065e4640 1 -- 172.21.15.16:0/371607679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8003f90 msgr2=0x7f0300157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.689+0000 7f03065e4640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8003f90 0x7f0300157400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f02f00119d0 tx=0x7f02f0011e90 comp rx=0 tx=0).stop 2024-09-12T01:34:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.689+0000 7f03065e4640 1 -- 172.21.15.16:0/371607679 shutdown_connections 2024-09-12T01:34:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.689+0000 7f03065e4640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f02ec06be30 0x7f02ec06e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.689+0000 7f03065e4640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0300155530 0x7f03001559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.689+0000 7f03065e4640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03000ce180 0x7f03001f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.689+0000 7f03065e4640 1 --2- 172.21.15.16:0/371607679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8003f90 0x7f0300157400 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.689+0000 7f03065e4640 1 -- 172.21.15.16:0/371607679 >> 172.21.15.16:0/371607679 conn(0x7f030006d1a0 msgr2=0x7f03000b96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.689+0000 7f03065e4640 1 -- 172.21.15.16:0/371607679 shutdown_connections 2024-09-12T01:34:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.689+0000 7f03065e4640 1 -- 172.21.15.16:0/371607679 wait complete. 2024-09-12T01:34:11.700 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-12T01:34:10.482954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592084,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":173,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":94,"acting":94,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":237376,"kb_used_data":4496,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737344,"statfs":{"total":773094113280,"available":772851040256,"internally_reserved":0,"allocated":4603904,"data_stored":2647351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1716,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.937377"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568397+0000","last_change":"2024-09-12T01:34:07.568217+0000","last_active":"2024-09-12T01:34:08.568397+0000","last_peered":"2024-09-12T01:34:08.568397+0000","last_clean":"2024-09-12T01:34:08.568397+0000","last_became_active":"2024-09-12T01:34:07.567979+0000","last_became_peered":"2024-09-12T01:34:07.567979+0000","last_unstale":"2024-09-12T01:34:08.568397+0000","last_undegraded":"2024-09-12T01:34:08.568397+0000","last_fullsized":"2024-09-12T01:34:08.568397+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:07.721403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567960+0000","last_change":"2024-09-12T01:34:07.566615+0000","last_active":"2024-09-12T01:34:08.567960+0000","last_peered":"2024-09-12T01:34:08.567960+0000","last_clean":"2024-09-12T01:34:08.567960+0000","last_became_active":"2024-09-12T01:34:07.566309+0000","last_became_peered":"2024-09-12T01:34:07.566309+0000","last_unstale":"2024-09-12T01:34:08.567960+0000","last_undegraded":"2024-09-12T01:34:08.567960+0000","last_fullsized":"2024-09-12T01:34:08.567960+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:10.428484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557777+0000","last_change":"2024-09-12T01:34:06.557777+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557777+0000","last_undegraded":"2024-09-12T01:34:06.557777+0000","last_fullsized":"2024-09-12T01:34:06.557777+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.576448+0000","last_change":"2024-09-12T01:34:07.574151+0000","last_active":"2024-09-12T01:34:07.576448+0000","last_peered":"2024-09-12T01:34:07.576448+0000","last_clean":"2024-09-12T01:34:07.576448+0000","last_became_active":"2024-09-12T01:34:07.573810+0000","last_became_peered":"2024-09-12T01:34:07.573810+0000","last_unstale":"2024-09-12T01:34:07.576448+0000","last_undegraded":"2024-09-12T01:34:07.576448+0000","last_fullsized":"2024-09-12T01:34:07.576448+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:56.562905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.577590+0000","last_change":"2024-09-12T01:34:07.572918+0000","last_active":"2024-09-12T01:34:07.577590+0000","last_peered":"2024-09-12T01:34:07.577590+0000","last_clean":"2024-09-12T01:34:07.577590+0000","last_became_active":"2024-09-12T01:34:07.572649+0000","last_became_peered":"2024-09-12T01:34:07.572649+0000","last_unstale":"2024-09-12T01:34:07.577590+0000","last_undegraded":"2024-09-12T01:34:07.577590+0000","last_fullsized":"2024-09-12T01:34:07.577590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:37.323845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.580009+0000","last_change":"2024-09-12T01:34:07.571121+0000","last_active":"2024-09-12T01:34:07.580009+0000","last_peered":"2024-09-12T01:34:07.580009+0000","last_clean":"2024-09-12T01:34:07.580009+0000","last_became_active":"2024-09-12T01:34:07.570676+0000","last_became_peered":"2024-09-12T01:34:07.570676+0000","last_unstale":"2024-09-12T01:34:07.580009+0000","last_undegraded":"2024-09-12T01:34:07.580009+0000","last_fullsized":"2024-09-12T01:34:07.580009+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:19.574669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567753+0000","last_change":"2024-09-12T01:34:07.581851+0000","last_active":"2024-09-12T01:34:08.567753+0000","last_peered":"2024-09-12T01:34:08.567753+0000","last_clean":"2024-09-12T01:34:08.567753+0000","last_became_active":"2024-09-12T01:34:07.581562+0000","last_became_peered":"2024-09-12T01:34:07.581562+0000","last_unstale":"2024-09-12T01:34:08.567753+0000","last_undegraded":"2024-09-12T01:34:08.567753+0000","last_fullsized":"2024-09-12T01:34:08.567753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.266349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.559288+0000","last_change":"2024-09-12T01:34:06.559288+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.559288+0000","last_undegraded":"2024-09-12T01:34:06.559288+0000","last_fullsized":"2024-09-12T01:34:06.559288+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.556833+0000","last_change":"2024-09-12T01:34:06.556833+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.556833+0000","last_undegraded":"2024-09-12T01:34:06.556833+0000","last_fullsized":"2024-09-12T01:34:06.556833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.574203+0000","last_change":"2024-09-12T01:34:07.570064+0000","last_active":"2024-09-12T01:34:07.574203+0000","last_peered":"2024-09-12T01:34:07.574203+0000","last_clean":"2024-09-12T01:34:07.574203+0000","last_became_active":"2024-09-12T01:34:07.569772+0000","last_became_peered":"2024-09-12T01:34:07.569772+0000","last_unstale":"2024-09-12T01:34:07.574203+0000","last_undegraded":"2024-09-12T01:34:07.574203+0000","last_fullsized":"2024-09-12T01:34:07.574203+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:21:38.197322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.574111+0000","last_change":"2024-09-12T01:34:07.570352+0000","last_active":"2024-09-12T01:34:07.574111+0000","last_peered":"2024-09-12T01:34:07.574111+0000","last_clean":"2024-09-12T01:34:07.574111+0000","last_became_active":"2024-09-12T01:34:07.569630+0000","last_became_peered":"2024-09-12T01:34:07.569630+0000","last_unstale":"2024-09-12T01:34:07.574111+0000","last_undegraded":"2024-09-12T01:34:07.574111+0000","last_fullsized":"2024-09-12T01:34:07.574111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:40.751596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584594+0000","last_change":"2024-09-12T01:34:07.583580+0000","last_active":"2024-09-12T01:34:07.584594+0000","last_peered":"2024-09-12T01:34:07.584594+0000","last_clean":"2024-09-12T01:34:07.584594+0000","last_became_active":"2024-09-12T01:34:07.580853+0000","last_became_peered":"2024-09-12T01:34:07.580853+0000","last_unstale":"2024-09-12T01:34:07.584594+0000","last_undegraded":"2024-09-12T01:34:07.584594+0000","last_fullsized":"2024-09-12T01:34:07.584594+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:21.696603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584661+0000","last_change":"2024-09-12T01:34:07.580355+0000","last_active":"2024-09-12T01:34:07.584661+0000","last_peered":"2024-09-12T01:34:07.584661+0000","last_clean":"2024-09-12T01:34:07.584661+0000","last_became_active":"2024-09-12T01:34:07.580020+0000","last_became_peered":"2024-09-12T01:34:07.580020+0000","last_unstale":"2024-09-12T01:34:07.584661+0000","last_undegraded":"2024-09-12T01:34:07.584661+0000","last_fullsized":"2024-09-12T01:34:07.584661+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:22.777538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584013+0000","last_change":"2024-09-12T01:34:07.581833+0000","last_active":"2024-09-12T01:34:07.584013+0000","last_peered":"2024-09-12T01:34:07.584013+0000","last_clean":"2024-09-12T01:34:07.584013+0000","last_became_active":"2024-09-12T01:34:07.581526+0000","last_became_peered":"2024-09-12T01:34:07.581526+0000","last_unstale":"2024-09-12T01:34:07.584013+0000","last_undegraded":"2024-09-12T01:34:07.584013+0000","last_fullsized":"2024-09-12T01:34:07.584013+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:28.448686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-12T01:34:06.573376+0000","last_change":"2024-09-12T01:34:06.572433+0000","last_active":"2024-09-12T01:34:06.573376+0000","last_peered":"2024-09-12T01:34:06.573376+0000","last_clean":"2024-09-12T01:34:06.573376+0000","last_became_active":"2024-09-12T01:34:06.572147+0000","last_became_peered":"2024-09-12T01:34:06.572147+0000","last_unstale":"2024-09-12T01:34:06.573376+0000","last_undegraded":"2024-09-12T01:34:06.573376+0000","last_fullsized":"2024-09-12T01:34:06.573376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:32.352391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.581274+0000","last_change":"2024-09-12T01:34:06.573185+0000","last_active":"2024-09-12T01:34:07.581274+0000","last_peered":"2024-09-12T01:34:07.581274+0000","last_clean":"2024-09-12T01:34:07.581274+0000","last_became_active":"2024-09-12T01:34:06.572814+0000","last_became_peered":"2024-09-12T01:34:06.572814+0000","last_unstale":"2024-09-12T01:34:07.581274+0000","last_undegraded":"2024-09-12T01:34:07.581274+0000","last_fullsized":"2024-09-12T01:34:07.581274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:35.191411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568251+0000","last_change":"2024-09-12T01:34:07.575202+0000","last_active":"2024-09-12T01:34:08.568251+0000","last_peered":"2024-09-12T01:34:08.568251+0000","last_clean":"2024-09-12T01:34:08.568251+0000","last_became_active":"2024-09-12T01:34:07.574888+0000","last_became_peered":"2024-09-12T01:34:07.574888+0000","last_unstale":"2024-09-12T01:34:08.568251+0000","last_undegraded":"2024-09-12T01:34:08.568251+0000","last_fullsized":"2024-09-12T01:34:08.568251+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:17.171969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.558243+0000","last_change":"2024-09-12T01:34:06.558243+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.558243+0000","last_undegraded":"2024-09-12T01:34:06.558243+0000","last_fullsized":"2024-09-12T01:34:06.558243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562765+0000","last_change":"2024-09-12T01:34:06.571529+0000","last_active":"2024-09-12T01:34:07.562765+0000","last_peered":"2024-09-12T01:34:07.562765+0000","last_clean":"2024-09-12T01:34:07.562765+0000","last_became_active":"2024-09-12T01:34:06.571192+0000","last_became_peered":"2024-09-12T01:34:06.571192+0000","last_unstale":"2024-09-12T01:34:07.562765+0000","last_undegraded":"2024-09-12T01:34:07.562765+0000","last_fullsized":"2024-09-12T01:34:07.562765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:18.183678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568020+0000","last_change":"2024-09-12T01:34:07.572915+0000","last_active":"2024-09-12T01:34:08.568020+0000","last_peered":"2024-09-12T01:34:08.568020+0000","last_clean":"2024-09-12T01:34:08.568020+0000","last_became_active":"2024-09-12T01:34:07.572585+0000","last_became_peered":"2024-09-12T01:34:07.572585+0000","last_unstale":"2024-09-12T01:34:08.568020+0000","last_undegraded":"2024-09-12T01:34:08.568020+0000","last_fullsized":"2024-09-12T01:34:08.568020+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:31.058724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.576425+0000","last_change":"2024-09-12T01:34:07.573978+0000","last_active":"2024-09-12T01:34:07.576425+0000","last_peered":"2024-09-12T01:34:07.576425+0000","last_clean":"2024-09-12T01:34:07.576425+0000","last_became_active":"2024-09-12T01:34:07.573412+0000","last_became_peered":"2024-09-12T01:34:07.573412+0000","last_unstale":"2024-09-12T01:34:07.576425+0000","last_undegraded":"2024-09-12T01:34:07.576425+0000","last_fullsized":"2024-09-12T01:34:07.576425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:05.655767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568445+0000","last_change":"2024-09-12T01:34:06.566440+0000","last_active":"2024-09-12T01:34:08.568445+0000","last_peered":"2024-09-12T01:34:08.568445+0000","last_clean":"2024-09-12T01:34:08.568445+0000","last_became_active":"2024-09-12T01:34:06.566046+0000","last_became_peered":"2024-09-12T01:34:06.566046+0000","last_unstale":"2024-09-12T01:34:08.568445+0000","last_undegraded":"2024-09-12T01:34:08.568445+0000","last_fullsized":"2024-09-12T01:34:08.568445+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:27.443857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557220+0000","last_change":"2024-09-12T01:34:06.557220+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557220+0000","last_undegraded":"2024-09-12T01:34:06.557220+0000","last_fullsized":"2024-09-12T01:34:06.557220+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568357+0000","last_change":"2024-09-12T01:34:06.576115+0000","last_active":"2024-09-12T01:34:08.568357+0000","last_peered":"2024-09-12T01:34:08.568357+0000","last_clean":"2024-09-12T01:34:08.568357+0000","last_became_active":"2024-09-12T01:34:06.575873+0000","last_became_peered":"2024-09-12T01:34:06.575873+0000","last_unstale":"2024-09-12T01:34:08.568357+0000","last_undegraded":"2024-09-12T01:34:08.568357+0000","last_fullsized":"2024-09-12T01:34:08.568357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.691775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557835+0000","last_change":"2024-09-12T01:34:06.557835+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557835+0000","last_undegraded":"2024-09-12T01:34:06.557835+0000","last_fullsized":"2024-09-12T01:34:06.557835+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568240+0000","last_change":"2024-09-12T01:34:06.566516+0000","last_active":"2024-09-12T01:34:08.568240+0000","last_peered":"2024-09-12T01:34:08.568240+0000","last_clean":"2024-09-12T01:34:08.568240+0000","last_became_active":"2024-09-12T01:34:06.566186+0000","last_became_peered":"2024-09-12T01:34:06.566186+0000","last_unstale":"2024-09-12T01:34:08.568240+0000","last_undegraded":"2024-09-12T01:34:08.568240+0000","last_fullsized":"2024-09-12T01:34:08.568240+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:59:30.145684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567848+0000","last_change":"2024-09-12T01:34:07.567627+0000","last_active":"2024-09-12T01:34:08.567848+0000","last_peered":"2024-09-12T01:34:08.567848+0000","last_clean":"2024-09-12T01:34:08.567848+0000","last_became_active":"2024-09-12T01:34:07.567381+0000","last_became_peered":"2024-09-12T01:34:07.567381+0000","last_unstale":"2024-09-12T01:34:08.567848+0000","last_undegraded":"2024-09-12T01:34:08.567848+0000","last_fullsized":"2024-09-12T01:34:08.567848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:30.783361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562642+0000","last_change":"2024-09-12T01:34:06.573250+0000","last_active":"2024-09-12T01:34:07.562642+0000","last_peered":"2024-09-12T01:34:07.562642+0000","last_clean":"2024-09-12T01:34:07.562642+0000","last_became_active":"2024-09-12T01:34:06.572929+0000","last_became_peered":"2024-09-12T01:34:06.572929+0000","last_unstale":"2024-09-12T01:34:07.562642+0000","last_undegraded":"2024-09-12T01:34:07.562642+0000","last_fullsized":"2024-09-12T01:34:07.562642+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:43.363051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568123+0000","last_change":"2024-09-12T01:34:07.566974+0000","last_active":"2024-09-12T01:34:08.568123+0000","last_peered":"2024-09-12T01:34:08.568123+0000","last_clean":"2024-09-12T01:34:08.568123+0000","last_became_active":"2024-09-12T01:34:07.566688+0000","last_became_peered":"2024-09-12T01:34:07.566688+0000","last_unstale":"2024-09-12T01:34:08.568123+0000","last_undegraded":"2024-09-12T01:34:08.568123+0000","last_fullsized":"2024-09-12T01:34:08.568123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:35.009027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568341+0000","last_change":"2024-09-12T01:34:07.576753+0000","last_active":"2024-09-12T01:34:08.568341+0000","last_peered":"2024-09-12T01:34:08.568341+0000","last_clean":"2024-09-12T01:34:08.568341+0000","last_became_active":"2024-09-12T01:34:07.576461+0000","last_became_peered":"2024-09-12T01:34:07.576461+0000","last_unstale":"2024-09-12T01:34:08.568341+0000","last_undegraded":"2024-09-12T01:34:08.568341+0000","last_fullsized":"2024-09-12T01:34:08.568341+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:12.831326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.589168+0000","last_change":"2024-09-12T01:34:06.572508+0000","last_active":"2024-09-12T01:34:07.589168+0000","last_peered":"2024-09-12T01:34:07.589168+0000","last_clean":"2024-09-12T01:34:07.589168+0000","last_became_active":"2024-09-12T01:34:06.572222+0000","last_became_peered":"2024-09-12T01:34:06.572222+0000","last_unstale":"2024-09-12T01:34:07.589168+0000","last_undegraded":"2024-09-12T01:34:07.589168+0000","last_fullsized":"2024-09-12T01:34:07.589168+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:12.959609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.574094+0000","last_change":"2024-09-12T01:34:07.571161+0000","last_active":"2024-09-12T01:34:07.574094+0000","last_peered":"2024-09-12T01:34:07.574094+0000","last_clean":"2024-09-12T01:34:07.574094+0000","last_became_active":"2024-09-12T01:34:07.570891+0000","last_became_peered":"2024-09-12T01:34:07.570891+0000","last_unstale":"2024-09-12T01:34:07.574094+0000","last_undegraded":"2024-09-12T01:34:07.574094+0000","last_fullsized":"2024-09-12T01:34:07.574094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:02.383752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562901+0000","last_change":"2024-09-12T01:34:06.574086+0000","last_active":"2024-09-12T01:34:07.562901+0000","last_peered":"2024-09-12T01:34:07.562901+0000","last_clean":"2024-09-12T01:34:07.562901+0000","last_became_active":"2024-09-12T01:34:06.573682+0000","last_became_peered":"2024-09-12T01:34:06.573682+0000","last_unstale":"2024-09-12T01:34:07.562901+0000","last_undegraded":"2024-09-12T01:34:07.562901+0000","last_fullsized":"2024-09-12T01:34:07.562901+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:38.064258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584434+0000","last_change":"2024-09-12T01:34:06.563081+0000","last_active":"2024-09-12T01:34:07.584434+0000","last_peered":"2024-09-12T01:34:07.584434+0000","last_clean":"2024-09-12T01:34:07.584434+0000","last_became_active":"2024-09-12T01:34:06.562728+0000","last_became_peered":"2024-09-12T01:34:06.562728+0000","last_unstale":"2024-09-12T01:34:07.584434+0000","last_undegraded":"2024-09-12T01:34:07.584434+0000","last_fullsized":"2024-09-12T01:34:07.584434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:56.551827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.558269+0000","last_change":"2024-09-12T01:34:06.558269+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.558269+0000","last_undegraded":"2024-09-12T01:34:06.558269+0000","last_fullsized":"2024-09-12T01:34:06.558269+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.560878+0000","last_change":"2024-09-12T01:34:06.573266+0000","last_active":"2024-09-12T01:34:07.560878+0000","last_peered":"2024-09-12T01:34:07.560878+0000","last_clean":"2024-09-12T01:34:07.560878+0000","last_became_active":"2024-09-12T01:34:06.572679+0000","last_became_peered":"2024-09-12T01:34:06.572679+0000","last_unstale":"2024-09-12T01:34:07.560878+0000","last_undegraded":"2024-09-12T01:34:07.560878+0000","last_fullsized":"2024-09-12T01:34:07.560878+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:50.930775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568150+0000","last_change":"2024-09-12T01:34:07.572416+0000","last_active":"2024-09-12T01:34:08.568150+0000","last_peered":"2024-09-12T01:34:08.568150+0000","last_clean":"2024-09-12T01:34:08.568150+0000","last_became_active":"2024-09-12T01:34:07.572092+0000","last_became_peered":"2024-09-12T01:34:07.572092+0000","last_unstale":"2024-09-12T01:34:08.568150+0000","last_undegraded":"2024-09-12T01:34:08.568150+0000","last_fullsized":"2024-09-12T01:34:08.568150+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:55.078476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.560460+0000","last_change":"2024-09-12T01:34:06.562339+0000","last_active":"2024-09-12T01:34:07.560460+0000","last_peered":"2024-09-12T01:34:07.560460+0000","last_clean":"2024-09-12T01:34:07.560460+0000","last_became_active":"2024-09-12T01:34:06.561990+0000","last_became_peered":"2024-09-12T01:34:06.561990+0000","last_unstale":"2024-09-12T01:34:07.560460+0000","last_undegraded":"2024-09-12T01:34:07.560460+0000","last_fullsized":"2024-09-12T01:34:07.560460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:09.175440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.583991+0000","last_change":"2024-09-12T01:34:07.581576+0000","last_active":"2024-09-12T01:34:07.583991+0000","last_peered":"2024-09-12T01:34:07.583991+0000","last_clean":"2024-09-12T01:34:07.583991+0000","last_became_active":"2024-09-12T01:34:07.580839+0000","last_became_peered":"2024-09-12T01:34:07.580839+0000","last_unstale":"2024-09-12T01:34:07.583991+0000","last_undegraded":"2024-09-12T01:34:07.583991+0000","last_fullsized":"2024-09-12T01:34:07.583991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:32.726522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.561435+0000","last_change":"2024-09-12T01:34:06.577637+0000","last_active":"2024-09-12T01:34:07.561435+0000","last_peered":"2024-09-12T01:34:07.561435+0000","last_clean":"2024-09-12T01:34:07.561435+0000","last_became_active":"2024-09-12T01:34:06.577338+0000","last_became_peered":"2024-09-12T01:34:06.577338+0000","last_unstale":"2024-09-12T01:34:07.561435+0000","last_undegraded":"2024-09-12T01:34:07.561435+0000","last_fullsized":"2024-09-12T01:34:07.561435+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:44.282477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567958+0000","last_change":"2024-09-12T01:34:07.563962+0000","last_active":"2024-09-12T01:34:08.567958+0000","last_peered":"2024-09-12T01:34:08.567958+0000","last_clean":"2024-09-12T01:34:08.567958+0000","last_became_active":"2024-09-12T01:34:07.563649+0000","last_became_peered":"2024-09-12T01:34:07.563649+0000","last_unstale":"2024-09-12T01:34:08.567958+0000","last_undegraded":"2024-09-12T01:34:08.567958+0000","last_fullsized":"2024-09-12T01:34:08.567958+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:36:19.605889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-12T01:34:06.566559+0000","last_change":"2024-09-12T01:34:06.564220+0000","last_active":"2024-09-12T01:34:06.566559+0000","last_peered":"2024-09-12T01:34:06.566559+0000","last_clean":"2024-09-12T01:34:06.566559+0000","last_became_active":"2024-09-12T01:34:06.563912+0000","last_became_peered":"2024-09-12T01:34:06.563912+0000","last_unstale":"2024-09-12T01:34:06.566559+0000","last_undegraded":"2024-09-12T01:34:06.566559+0000","last_fullsized":"2024-09-12T01:34:06.566559+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:26.470501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557400+0000","last_change":"2024-09-12T01:34:06.557400+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557400+0000","last_undegraded":"2024-09-12T01:34:06.557400+0000","last_fullsized":"2024-09-12T01:34:06.557400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-12T01:34:06.569286+0000","last_change":"2024-09-12T01:34:06.567684+0000","last_active":"2024-09-12T01:34:06.569286+0000","last_peered":"2024-09-12T01:34:06.569286+0000","last_clean":"2024-09-12T01:34:06.569286+0000","last_became_active":"2024-09-12T01:34:06.567184+0000","last_became_peered":"2024-09-12T01:34:06.567184+0000","last_unstale":"2024-09-12T01:34:06.569286+0000","last_undegraded":"2024-09-12T01:34:06.569286+0000","last_fullsized":"2024-09-12T01:34:06.569286+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:23.197060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.580895+0000","last_change":"2024-09-12T01:34:07.573865+0000","last_active":"2024-09-12T01:34:07.580895+0000","last_peered":"2024-09-12T01:34:07.580895+0000","last_clean":"2024-09-12T01:34:07.580895+0000","last_became_active":"2024-09-12T01:34:07.573578+0000","last_became_peered":"2024-09-12T01:34:07.573578+0000","last_unstale":"2024-09-12T01:34:07.580895+0000","last_undegraded":"2024-09-12T01:34:07.580895+0000","last_fullsized":"2024-09-12T01:34:07.580895+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:03.055098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567833+0000","last_change":"2024-09-12T01:34:06.566830+0000","last_active":"2024-09-12T01:34:08.567833+0000","last_peered":"2024-09-12T01:34:08.567833+0000","last_clean":"2024-09-12T01:34:08.567833+0000","last_became_active":"2024-09-12T01:34:06.566321+0000","last_became_peered":"2024-09-12T01:34:06.566321+0000","last_unstale":"2024-09-12T01:34:08.567833+0000","last_undegraded":"2024-09-12T01:34:08.567833+0000","last_fullsized":"2024-09-12T01:34:08.567833+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:31.718104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-12T01:34:02.259557+0000","last_change":"2024-09-12T01:33:01.096542+0000","last_active":"2024-09-12T01:34:02.259557+0000","last_peered":"2024-09-12T01:34:02.259557+0000","last_clean":"2024-09-12T01:34:02.259557+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:34:02.259557+0000","last_undegraded":"2024-09-12T01:34:02.259557+0000","last_fullsized":"2024-09-12T01:34:02.259557+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_clean_scrub_stamp":"2024-09-12T01:32:58.512553+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:48:58.741813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":60,"acting":60,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1716,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":32,"acting":32,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":999424,"data_stored":763507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":173139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":173139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":173139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":471040,"data_stored":201520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":446464,"data_stored":193272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":446464,"data_stored":193252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1024000,"data_stored":776383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:34:11.703 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3d is not active+clean 2024-09-12T01:34:11.703 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-12T01:34:06.557777+0000', 'last_change': '2024-09-12T01:34:06.557777+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.557777+0000', 'last_undegraded': '2024-09-12T01:34:06.557777+0000', 'last_fullsized': '2024-09-12T01:34:06.557777+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T01:34:11.704 INFO:tasks.ceph.ceph_manager.ceph:PG 6.39 is not active+clean 2024-09-12T01:34:11.704 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.704 INFO:tasks.ceph.ceph_manager.ceph:PG 6.37 is not active+clean 2024-09-12T01:34:11.704 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-12T01:34:06.559288+0000', 'last_change': '2024-09-12T01:34:06.559288+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.559288+0000', 'last_undegraded': '2024-09-12T01:34:06.559288+0000', 'last_fullsized': '2024-09-12T01:34:06.559288+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T01:34:11.704 INFO:tasks.ceph.ceph_manager.ceph:PG 6.36 is not active+clean 2024-09-12T01:34:11.704 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.704 INFO:tasks.ceph.ceph_manager.ceph:PG 6.35 is not active+clean 2024-09-12T01:34:11.704 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.704 INFO:tasks.ceph.ceph_manager.ceph:PG 6.34 is not active+clean 2024-09-12T01:34:11.704 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-12T01:34:06.556833+0000', 'last_change': '2024-09-12T01:34:06.556833+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.556833+0000', 'last_undegraded': '2024-09-12T01:34:06.556833+0000', 'last_fullsized': '2024-09-12T01:34:06.556833+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T01:34:11.704 INFO:tasks.ceph.ceph_manager.ceph:PG 6.33 is not active+clean 2024-09-12T01:34:11.705 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.705 INFO:tasks.ceph.ceph_manager.ceph:PG 6.32 is not active+clean 2024-09-12T01:34:11.705 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.705 INFO:tasks.ceph.ceph_manager.ceph:PG 6.31 is not active+clean 2024-09-12T01:34:11.705 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.705 INFO:tasks.ceph.ceph_manager.ceph:PG 6.30 is not active+clean 2024-09-12T01:34:11.705 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.705 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2f is not active+clean 2024-09-12T01:34:11.705 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.705 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2e is not active+clean 2024-09-12T01:34:11.705 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.706 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2c is not active+clean 2024-09-12T01:34:11.706 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.706 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2b is not active+clean 2024-09-12T01:34:11.706 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.706 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2a is not active+clean 2024-09-12T01:34:11.706 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.706 INFO:tasks.ceph.ceph_manager.ceph:PG 6.29 is not active+clean 2024-09-12T01:34:11.706 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.706 INFO:tasks.ceph.ceph_manager.ceph:PG 6.27 is not active+clean 2024-09-12T01:34:11.706 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.706 INFO:tasks.ceph.ceph_manager.ceph:PG 6.25 is not active+clean 2024-09-12T01:34:11.706 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.707 INFO:tasks.ceph.ceph_manager.ceph:PG 6.24 is not active+clean 2024-09-12T01:34:11.707 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.707 INFO:tasks.ceph.ceph_manager.ceph:PG 6.23 is not active+clean 2024-09-12T01:34:11.707 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.707 INFO:tasks.ceph.ceph_manager.ceph:PG 6.22 is not active+clean 2024-09-12T01:34:11.707 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.707 INFO:tasks.ceph.ceph_manager.ceph:PG 6.21 is not active+clean 2024-09-12T01:34:11.707 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.707 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1c is not active+clean 2024-09-12T01:34:11.707 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.707 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1f is not active+clean 2024-09-12T01:34:11.708 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.708 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1e is not active+clean 2024-09-12T01:34:11.708 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.708 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1c is not active+clean 2024-09-12T01:34:11.708 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.708 INFO:tasks.ceph.ceph_manager.ceph:PG 6.18 is not active+clean 2024-09-12T01:34:11.708 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-12T01:34:06.558243+0000', 'last_change': '2024-09-12T01:34:06.558243+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.558243+0000', 'last_undegraded': '2024-09-12T01:34:06.558243+0000', 'last_fullsized': '2024-09-12T01:34:06.558243+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T01:34:11.708 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1b is not active+clean 2024-09-12T01:34:11.708 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.708 INFO:tasks.ceph.ceph_manager.ceph:PG 6.19 is not active+clean 2024-09-12T01:34:11.708 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.709 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1a is not active+clean 2024-09-12T01:34:11.709 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.709 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1a is not active+clean 2024-09-12T01:34:11.709 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.709 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-09-12T01:34:11.709 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.709 INFO:tasks.ceph.ceph_manager.ceph:PG 6.15 is not active+clean 2024-09-12T01:34:11.709 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.709 INFO:tasks.ceph.ceph_manager.ceph:PG 5.16 is not active+clean 2024-09-12T01:34:11.709 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.709 INFO:tasks.ceph.ceph_manager.ceph:PG 6.16 is not active+clean 2024-09-12T01:34:11.710 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-12T01:34:06.557220+0000', 'last_change': '2024-09-12T01:34:06.557220+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.557220+0000', 'last_undegraded': '2024-09-12T01:34:06.557220+0000', 'last_fullsized': '2024-09-12T01:34:06.557220+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T01:34:11.710 INFO:tasks.ceph.ceph_manager.ceph:PG 5.15 is not active+clean 2024-09-12T01:34:11.710 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.710 INFO:tasks.ceph.ceph_manager.ceph:PG 6.17 is not active+clean 2024-09-12T01:34:11.710 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.710 INFO:tasks.ceph.ceph_manager.ceph:PG 6.10 is not active+clean 2024-09-12T01:34:11.710 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-12T01:34:06.557835+0000', 'last_change': '2024-09-12T01:34:06.557835+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.557835+0000', 'last_undegraded': '2024-09-12T01:34:06.557835+0000', 'last_fullsized': '2024-09-12T01:34:06.557835+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T01:34:11.710 INFO:tasks.ceph.ceph_manager.ceph:PG 6.12 is not active+clean 2024-09-12T01:34:11.710 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.710 INFO:tasks.ceph.ceph_manager.ceph:PG 5.11 is not active+clean 2024-09-12T01:34:11.710 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.711 INFO:tasks.ceph.ceph_manager.ceph:PG 5.10 is not active+clean 2024-09-12T01:34:11.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.711 INFO:tasks.ceph.ceph_manager.ceph:PG 5.f is not active+clean 2024-09-12T01:34:11.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.711 INFO:tasks.ceph.ceph_manager.ceph:PG 6.d is not active+clean 2024-09-12T01:34:11.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.711 INFO:tasks.ceph.ceph_manager.ceph:PG 5.e is not active+clean 2024-09-12T01:34:11.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.711 INFO:tasks.ceph.ceph_manager.ceph:PG 6.e is not active+clean 2024-09-12T01:34:11.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.711 INFO:tasks.ceph.ceph_manager.ceph:PG 6.8 is not active+clean 2024-09-12T01:34:11.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.712 INFO:tasks.ceph.ceph_manager.ceph:PG 6.9 is not active+clean 2024-09-12T01:34:11.712 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-12T01:34:06.558269+0000', 'last_change': '2024-09-12T01:34:06.558269+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.558269+0000', 'last_undegraded': '2024-09-12T01:34:06.558269+0000', 'last_fullsized': '2024-09-12T01:34:06.558269+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T01:34:11.712 INFO:tasks.ceph.ceph_manager.ceph:PG 6.a is not active+clean 2024-09-12T01:34:11.712 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.712 INFO:tasks.ceph.ceph_manager.ceph:PG 5.9 is not active+clean 2024-09-12T01:34:11.712 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.712 INFO:tasks.ceph.ceph_manager.ceph:PG 5.7 is not active+clean 2024-09-12T01:34:11.712 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.712 INFO:tasks.ceph.ceph_manager.ceph:PG 6.5 is not active+clean 2024-09-12T01:34:11.712 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.713 INFO:tasks.ceph.ceph_manager.ceph:PG 6.0 is not active+clean 2024-09-12T01:34:11.713 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-12T01:34:06.557400+0000', 'last_change': '2024-09-12T01:34:06.557400+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.557400+0000', 'last_undegraded': '2024-09-12T01:34:06.557400+0000', 'last_fullsized': '2024-09-12T01:34:06.557400+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T01:34:11.713 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2 is not active+clean 2024-09-12T01:34:11.713 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.713 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2 is not active+clean 2024-09-12T01:34:11.713 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.713 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1 is not active+clean 2024-09-12T01:34:11.713 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.713 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3 is not active+clean 2024-09-12T01:34:11.713 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.713 INFO:tasks.ceph.ceph_manager.ceph:PG 6.7 is not active+clean 2024-09-12T01:34:11.714 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:06.545580+0000', 'last_change': '2024-09-12T01:34:06.545580+0000', 'last_active': '2024-09-12T01:34:06.545580+0000', 'last_peered': '2024-09-12T01:34:06.545580+0000', 'last_clean': '2024-09-12T01:34:06.545580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:06.545580+0000', 'last_undegraded': '2024-09-12T01:34:06.545580+0000', 'last_fullsized': '2024-09-12T01:34:06.545580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:06.545580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.714 INFO:tasks.ceph.ceph_manager.ceph:PG 5.4 is not active+clean 2024-09-12T01:34:11.714 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T01:34:05.540751+0000', 'last_change': '2024-09-12T01:34:05.540751+0000', 'last_active': '2024-09-12T01:34:05.540751+0000', 'last_peered': '2024-09-12T01:34:05.540751+0000', 'last_clean': '2024-09-12T01:34:05.540751+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T01:34:05.540751+0000', 'last_undegraded': '2024-09-12T01:34:05.540751+0000', 'last_fullsized': '2024-09-12T01:34:05.540751+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'last_clean_scrub_stamp': '2024-09-12T01:34:05.540751+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T01:34:11.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-12T01:34:11.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.842+0000 7f2e603fd640 1 Processor -- start 2024-09-12T01:34:11.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.842+0000 7f2e603fd640 1 -- start start 2024-09-12T01:34:11.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.843+0000 7f2e603fd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e5815e560 0x7f2e580ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.843+0000 7f2e603fd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e580d1550 0x7f2e580b7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.843+0000 7f2e603fd640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e580d3480 con 0x7f2e580cf510 2024-09-12T01:34:11.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.843+0000 7f2e603fd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2e580d3760 con 0x7f2e5815e560 2024-09-12T01:34:11.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.843+0000 7f2e603fd640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f2e580d3a40 con 0x7f2e580d1550 2024-09-12T01:34:11.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.843+0000 7f2e5e199640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e5815e560 0x7f2e580ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.843+0000 7f2e5e199640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e5815e560 0x7f2e580ceee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41446/0 (socket says 172.21.15.16:41446) 2024-09-12T01:34:11.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.843+0000 7f2e5e199640 1 -- 172.21.15.16:0/1819312484 learned_addr learned my addr 172.21.15.16:0/1819312484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:11.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.843+0000 7f2e5e99a640 1 --2- 172.21.15.16:0/1819312484 >> v2:172.21.15.33:3300/0 conn(0x7f2e580d1550 0x7f2e580b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.843+0000 7f2e5e199640 1 -- 172.21.15.16:0/1819312484 >> v2:172.21.15.33:3300/0 conn(0x7f2e580d1550 msgr2=0x7f2e580b7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:11.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.843+0000 7f2e5e199640 1 --2- 172.21.15.16:0/1819312484 >> v2:172.21.15.33:3300/0 conn(0x7f2e580d1550 0x7f2e580b7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.843+0000 7f2e5e199640 1 -- 172.21.15.16:0/1819312484 >> v1:172.21.15.33:6789/0 conn(0x7f2e580cf510 legacy=0x7f2e580d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.843+0000 7f2e5e199640 1 -- 172.21.15.16:0/1819312484 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e580d3d20 con 0x7f2e5815e560 2024-09-12T01:34:11.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.844+0000 7f2e5e199640 1 --2- 172.21.15.16:0/1819312484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5815e560 0x7f2e580ceee0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2e580b76e0 tx=0x7f2e5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54797e03969b94c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:11.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.844+0000 7f2e5d197640 1 -- 172.21.15.16:0/1819312484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54037070 con 0x7f2e5815e560 2024-09-12T01:34:11.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.844+0000 7f2e5d197640 1 -- 172.21.15.16:0/1819312484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e54032440 con 0x7f2e5815e560 2024-09-12T01:34:11.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.844+0000 7f2e5d197640 1 -- 172.21.15.16:0/1819312484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54032810 con 0x7f2e5815e560 2024-09-12T01:34:11.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.845+0000 7f2e603fd640 1 -- 172.21.15.16:0/1819312484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5815e560 msgr2=0x7f2e580ceee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.845+0000 7f2e603fd640 1 --2- 172.21.15.16:0/1819312484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5815e560 0x7f2e580ceee0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2e580b76e0 tx=0x7f2e5402a270 comp rx=0 tx=0).stop 2024-09-12T01:34:11.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.845+0000 7f2e603fd640 1 -- 172.21.15.16:0/1819312484 shutdown_connections 2024-09-12T01:34:11.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.845+0000 7f2e603fd640 1 --2- 172.21.15.16:0/1819312484 >> v2:172.21.15.33:3300/0 conn(0x7f2e580d1550 0x7f2e580b7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.845+0000 7f2e603fd640 1 --2- 172.21.15.16:0/1819312484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5815e560 0x7f2e580ceee0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.845+0000 7f2e603fd640 1 -- 172.21.15.16:0/1819312484 >> 172.21.15.16:0/1819312484 conn(0x7f2e58073850 msgr2=0x7f2e5806cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:11.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.845+0000 7f2e603fd640 1 -- 172.21.15.16:0/1819312484 shutdown_connections 2024-09-12T01:34:11.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.845+0000 7f2e603fd640 1 -- 172.21.15.16:0/1819312484 wait complete. 2024-09-12T01:34:11.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.846+0000 7f2e603fd640 1 Processor -- start 2024-09-12T01:34:11.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.846+0000 7f2e603fd640 1 -- start start 2024-09-12T01:34:11.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e603fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e580cf510 0x7f2e58160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e603fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e580d1550 0x7f2e580cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e603fd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2e5815e560 0x7f2e58161240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e603fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5815eac0 con 0x7f2e580cf510 2024-09-12T01:34:11.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e603fd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5815e940 con 0x7f2e580d1550 2024-09-12T01:34:11.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e603fd640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f2e580cf980 con 0x7f2e5815e560 2024-09-12T01:34:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e5e199640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e580cf510 0x7f2e58160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e5e199640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e580cf510 0x7f2e58160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41450/0 (socket says 172.21.15.16:41450) 2024-09-12T01:34:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e5e199640 1 -- 172.21.15.16:0/3963864698 learned_addr learned my addr 172.21.15.16:0/3963864698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e5d998640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e580d1550 0x7f2e580cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e5e99a640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2e5815e560 0x7f2e58161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e5e199640 1 -- 172.21.15.16:0/3963864698 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2e5815e560 msgr2=0x7f2e58161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e5e199640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2e5815e560 0x7f2e58161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e5e199640 1 -- 172.21.15.16:0/3963864698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e580d1550 msgr2=0x7f2e580cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e5e199640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e580d1550 0x7f2e580cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e5e199640 1 -- 172.21.15.16:0/3963864698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e58160a90 con 0x7f2e580cf510 2024-09-12T01:34:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e5e199640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e580cf510 0x7f2e58160550 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2e5402a2b0 tx=0x7f2e540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.847+0000 7f2e4effd640 1 -- 172.21.15.16:0/3963864698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54041020 con 0x7f2e580cf510 2024-09-12T01:34:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.848+0000 7f2e4effd640 1 -- 172.21.15.16:0/3963864698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e540306c0 con 0x7f2e580cf510 2024-09-12T01:34:11.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.848+0000 7f2e4effd640 1 -- 172.21.15.16:0/3963864698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54037040 con 0x7f2e580cf510 2024-09-12T01:34:11.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.848+0000 7f2e603fd640 1 -- 172.21.15.16:0/3963864698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e580ccfb0 con 0x7f2e580cf510 2024-09-12T01:34:11.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.848+0000 7f2e603fd640 1 -- 172.21.15.16:0/3963864698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e580cd570 con 0x7f2e580cf510 2024-09-12T01:34:11.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.848+0000 7f2e603fd640 1 -- 172.21.15.16:0/3963864698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e580ceee0 con 0x7f2e580cf510 2024-09-12T01:34:11.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.849+0000 7f2e4effd640 1 -- 172.21.15.16:0/3963864698 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e54044020 con 0x7f2e580cf510 2024-09-12T01:34:11.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.849+0000 7f2e4effd640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2e3c06be30 0x7f2e3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:11.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.850+0000 7f2e4effd640 1 -- 172.21.15.16:0/3963864698 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2e540ab830 con 0x7f2e580cf510 2024-09-12T01:34:11.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.850+0000 7f2e5d998640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2e3c06be30 0x7f2e3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:11.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.850+0000 7f2e5d998640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2e3c06be30 0x7f2e3c06e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2e580b83b0 tx=0x7f2e48009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:11.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:11.854+0000 7f2e4effd640 1 -- 172.21.15.16:0/3963864698 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2e5404b030 con 0x7f2e580cf510 2024-09-12T01:34:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.095+0000 7f2e603fd640 1 -- 172.21.15.16:0/3963864698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f2e5815c780 con 0x7f2e580cf510 2024-09-12T01:34:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.095+0000 7f2e4effd640 1 -- 172.21.15.16:0/3963864698 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1764 (secure 0 0 0) 0x7f2e5407a050 con 0x7f2e580cf510 2024-09-12T01:34:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:12.095 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":83,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":40,"num_osds":8,"num_up_osds":8,"osd_up_since":1726104779,"num_in_osds":8,"osd_in_since":1726104770,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":50},{"state_name":"active+clean","count":39},{"state_name":"creating+peering","count":8}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":592084,"bytes_used":243073024,"bytes_avail":772851040256,"bytes_total":773094113280,"unknown_pgs_ratio":0.51546388864517212,"inactive_pgs_ratio":0.082474224269390106,"write_bytes_sec":2281,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":16,"btime":"2024-09-12T01:34:10:019140+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4932}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-12T01:34:02.481827+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-12T01:34:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.097+0000 7f2e603fd640 1 -- 172.21.15.16:0/3963864698 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2e3c06be30 msgr2=0x7f2e3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.097+0000 7f2e603fd640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2e3c06be30 0x7f2e3c06e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2e580b83b0 tx=0x7f2e48009040 comp rx=0 tx=0).stop 2024-09-12T01:34:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.097+0000 7f2e603fd640 1 -- 172.21.15.16:0/3963864698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e580cf510 msgr2=0x7f2e58160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.097+0000 7f2e603fd640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e580cf510 0x7f2e58160550 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2e5402a2b0 tx=0x7f2e540046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.097+0000 7f2e603fd640 1 -- 172.21.15.16:0/3963864698 shutdown_connections 2024-09-12T01:34:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.097+0000 7f2e603fd640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2e3c06be30 0x7f2e3c06e2f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.097+0000 7f2e603fd640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2e5815e560 0x7f2e58161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.097+0000 7f2e603fd640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e580d1550 0x7f2e580cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.097+0000 7f2e603fd640 1 --2- 172.21.15.16:0/3963864698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e580cf510 0x7f2e58160550 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.097+0000 7f2e603fd640 1 -- 172.21.15.16:0/3963864698 >> 172.21.15.16:0/3963864698 conn(0x7f2e58073850 msgr2=0x7f2e580b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.097+0000 7f2e603fd640 1 -- 172.21.15.16:0/3963864698 shutdown_connections 2024-09-12T01:34:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.097+0000 7f2e603fd640 1 -- 172.21.15.16:0/3963864698 wait complete. 2024-09-12T01:34:12.107 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-12T01:34:12.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:34:12.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.210+0000 7fbd0095f640 1 Processor -- start 2024-09-12T01:34:12.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.210+0000 7fbd0095f640 1 -- start start 2024-09-12T01:34:12.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.211+0000 7fbd0095f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbcfc145080 0x7fbcfc0ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:12.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.211+0000 7fbd0095f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbcfc145460 0x7fbcfc0d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.211+0000 7fbd0095f640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcfc145a70 con 0x7fbcfc0af810 2024-09-12T01:34:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.211+0000 7fbd0095f640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fbcfc145d50 con 0x7fbcfc145460 2024-09-12T01:34:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.211+0000 7fbd0095f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fbcfc0d1160 con 0x7fbcfc145080 2024-09-12T01:34:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.211+0000 7fbcfa59c640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbcfc145080 0x7fbcfc0ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.211+0000 7fbcf9d9b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbcfc145460 0x7fbcfc0d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.211+0000 7fbcfa59c640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbcfc145080 0x7fbcfc0ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:39320/0 (socket says 172.21.15.16:39320) 2024-09-12T01:34:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.211+0000 7fbcfa59c640 1 -- 172.21.15.16:0/196504689 learned_addr learned my addr 172.21.15.16:0/196504689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.211+0000 7fbcfad9d640 1 --1- 172.21.15.16:0/196504689 >> v1:172.21.15.33:6789/0 conn(0x7fbcfc0af810 0x7fbcfc0afcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:58634/0 (socket says 172.21.15.16:58634) 2024-09-12T01:34:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.212+0000 7fbcf9d9b640 1 -- 172.21.15.16:0/196504689 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbcfc145080 msgr2=0x7fbcfc0ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.212+0000 7fbcf9d9b640 1 --2- 172.21.15.16:0/196504689 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbcfc145080 0x7fbcfc0ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.212+0000 7fbcf9d9b640 1 -- 172.21.15.16:0/196504689 >> v1:172.21.15.33:6789/0 conn(0x7fbcfc0af810 legacy=0x7fbcfc0afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.212+0000 7fbcf959a640 1 -- 172.21.15.16:0/196504689 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3132824557 0 0) 0x7fbcfc145a70 con 0x7fbcfc0af810 2024-09-12T01:34:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.212+0000 7fbcf9d9b640 1 -- 172.21.15.16:0/196504689 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc0d1440 con 0x7fbcfc145460 2024-09-12T01:34:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.212+0000 7fbcfa59c640 1 --2- 172.21.15.16:0/196504689 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbcfc145080 0x7fbcfc0ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.213+0000 7fbcf9d9b640 1 --2- 172.21.15.16:0/196504689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcfc145460 0x7fbcfc0d1960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbcf00097a0 tx=0x7fbcf002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb0ebda51d7be11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.213+0000 7fbcf959a640 1 -- 172.21.15.16:0/196504689 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0037070 con 0x7fbcfc145460 2024-09-12T01:34:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.213+0000 7fbcf959a640 1 -- 172.21.15.16:0/196504689 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf0032440 con 0x7fbcfc145460 2024-09-12T01:34:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.213+0000 7fbcf959a640 1 -- 172.21.15.16:0/196504689 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0032810 con 0x7fbcfc145460 2024-09-12T01:34:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.213+0000 7fbd0095f640 1 -- 172.21.15.16:0/196504689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcfc145460 msgr2=0x7fbcfc0d1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.213+0000 7fbd0095f640 1 --2- 172.21.15.16:0/196504689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcfc145460 0x7fbcfc0d1960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbcf00097a0 tx=0x7fbcf002a180 comp rx=0 tx=0).stop 2024-09-12T01:34:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.213+0000 7fbd0095f640 1 -- 172.21.15.16:0/196504689 shutdown_connections 2024-09-12T01:34:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.213+0000 7fbd0095f640 1 --2- 172.21.15.16:0/196504689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcfc145460 0x7fbcfc0d1960 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.213+0000 7fbd0095f640 1 --2- 172.21.15.16:0/196504689 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbcfc145080 0x7fbcfc0ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.213+0000 7fbd0095f640 1 -- 172.21.15.16:0/196504689 >> 172.21.15.16:0/196504689 conn(0x7fbcfc0b6570 msgr2=0x7fbcfc0b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.213+0000 7fbd0095f640 1 -- 172.21.15.16:0/196504689 shutdown_connections 2024-09-12T01:34:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.213+0000 7fbd0095f640 1 -- 172.21.15.16:0/196504689 wait complete. 2024-09-12T01:34:12.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.215+0000 7fbd0095f640 1 Processor -- start 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.215+0000 7fbd0095f640 1 -- start start 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbd0095f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbcfc0af810 0x7fbcfc15b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbd0095f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc145080 0x7fbcfc0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbd0095f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcfc145460 0x7fbcfc13ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbd0095f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc15fcb0 con 0x7fbcfc145080 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbd0095f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc0b03d0 con 0x7fbcfc145460 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbd0095f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fbcfc15fe30 con 0x7fbcfc0af810 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbcf9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc145080 0x7fbcfc0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbcf9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc145080 0x7fbcfc0cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41466/0 (socket says 172.21.15.16:41466) 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbcf9d9b640 1 -- 172.21.15.16:0/2242470944 learned_addr learned my addr 172.21.15.16:0/2242470944 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbcfa59c640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbcfc0af810 0x7fbcfc15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbcfad9d640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcfc145460 0x7fbcfc13ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbcf9d9b640 1 -- 172.21.15.16:0/2242470944 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbcfc0af810 msgr2=0x7fbcfc15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbcf9d9b640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbcfc0af810 0x7fbcfc15b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbcf9d9b640 1 -- 172.21.15.16:0/2242470944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcfc145460 msgr2=0x7fbcfc13ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbcf9d9b640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcfc145460 0x7fbcfc13ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbcf9d9b640 1 -- 172.21.15.16:0/2242470944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc0cb1b0 con 0x7fbcfc145080 2024-09-12T01:34:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbcf9d9b640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc145080 0x7fbcfc0cc440 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbcf00097a0 tx=0x7fbcf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbceaffd640 1 -- 172.21.15.16:0/2242470944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0037040 con 0x7fbcfc145080 2024-09-12T01:34:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbceaffd640 1 -- 172.21.15.16:0/2242470944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf0004230 con 0x7fbcfc145080 2024-09-12T01:34:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbd0095f640 1 -- 172.21.15.16:0/2242470944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcfc0cb960 con 0x7fbcfc145080 2024-09-12T01:34:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.216+0000 7fbceaffd640 1 -- 172.21.15.16:0/2242470944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0003610 con 0x7fbcfc145080 2024-09-12T01:34:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.217+0000 7fbd0095f640 1 -- 172.21.15.16:0/2242470944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcfc15bb50 con 0x7fbcfc145080 2024-09-12T01:34:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.218+0000 7fbd0095f640 1 -- 172.21.15.16:0/2242470944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcfc0cb760 con 0x7fbcfc145080 2024-09-12T01:34:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.218+0000 7fbceaffd640 1 -- 172.21.15.16:0/2242470944 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbcf00037b0 con 0x7fbcfc145080 2024-09-12T01:34:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.219+0000 7fbceaffd640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.219+0000 7fbcfa59c640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.219+0000 7fbceaffd640 1 -- 172.21.15.16:0/2242470944 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fbcf00ac000 con 0x7fbcfc145080 2024-09-12T01:34:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.219+0000 7fbcfa59c640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbcec006ca0 tx=0x7fbcec009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:12.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.223+0000 7fbceaffd640 1 -- 172.21.15.16:0/2242470944 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbcf007a2f0 con 0x7fbcfc145080 2024-09-12T01:34:12.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.396+0000 7fbd0095f640 1 -- 172.21.15.16:0/2242470944 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbcfc15c520 con 0x7fbcdc06be30 2024-09-12T01:34:12.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.402+0000 7fbceaffd640 1 -- 172.21.15.16:0/2242470944 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243699 (secure 0 0 0) 0x7fbcfc15c520 con 0x7fbcdc06be30 2024-09-12T01:34:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:12.405 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:34:12.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.405+0000 7fbd0095f640 1 -- 172.21.15.16:0/2242470944 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbcdc06be30 msgr2=0x7fbcdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.405+0000 7fbd0095f640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbcec006ca0 tx=0x7fbcec009240 comp rx=0 tx=0).stop 2024-09-12T01:34:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.405+0000 7fbd0095f640 1 -- 172.21.15.16:0/2242470944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc145080 msgr2=0x7fbcfc0cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.405+0000 7fbd0095f640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc145080 0x7fbcfc0cc440 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbcf00097a0 tx=0x7fbcf00046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.405+0000 7fbd0095f640 1 -- 172.21.15.16:0/2242470944 shutdown_connections 2024-09-12T01:34:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.405+0000 7fbd0095f640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.405+0000 7fbd0095f640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcfc145460 0x7fbcfc13ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.405+0000 7fbd0095f640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc145080 0x7fbcfc0cc440 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.405+0000 7fbd0095f640 1 --2- 172.21.15.16:0/2242470944 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbcfc0af810 0x7fbcfc15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.405+0000 7fbd0095f640 1 -- 172.21.15.16:0/2242470944 >> 172.21.15.16:0/2242470944 conn(0x7fbcfc0b6570 msgr2=0x7fbcfc0b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.405+0000 7fbd0095f640 1 -- 172.21.15.16:0/2242470944 shutdown_connections 2024-09-12T01:34:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:12.405+0000 7fbd0095f640 1 -- 172.21.15.16:0/2242470944 wait complete. 2024-09-12T01:34:12.416 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-12T01:34:10.482954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592084,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":173,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":94,"acting":94,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":237376,"kb_used_data":4496,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737344,"statfs":{"total":773094113280,"available":772851040256,"internally_reserved":0,"allocated":4603904,"data_stored":2647351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1716,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.937377"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568397+0000","last_change":"2024-09-12T01:34:07.568217+0000","last_active":"2024-09-12T01:34:08.568397+0000","last_peered":"2024-09-12T01:34:08.568397+0000","last_clean":"2024-09-12T01:34:08.568397+0000","last_became_active":"2024-09-12T01:34:07.567979+0000","last_became_peered":"2024-09-12T01:34:07.567979+0000","last_unstale":"2024-09-12T01:34:08.568397+0000","last_undegraded":"2024-09-12T01:34:08.568397+0000","last_fullsized":"2024-09-12T01:34:08.568397+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:07.721403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567960+0000","last_change":"2024-09-12T01:34:07.566615+0000","last_active":"2024-09-12T01:34:08.567960+0000","last_peered":"2024-09-12T01:34:08.567960+0000","last_clean":"2024-09-12T01:34:08.567960+0000","last_became_active":"2024-09-12T01:34:07.566309+0000","last_became_peered":"2024-09-12T01:34:07.566309+0000","last_unstale":"2024-09-12T01:34:08.567960+0000","last_undegraded":"2024-09-12T01:34:08.567960+0000","last_fullsized":"2024-09-12T01:34:08.567960+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:10.428484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557777+0000","last_change":"2024-09-12T01:34:06.557777+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557777+0000","last_undegraded":"2024-09-12T01:34:06.557777+0000","last_fullsized":"2024-09-12T01:34:06.557777+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.576448+0000","last_change":"2024-09-12T01:34:07.574151+0000","last_active":"2024-09-12T01:34:07.576448+0000","last_peered":"2024-09-12T01:34:07.576448+0000","last_clean":"2024-09-12T01:34:07.576448+0000","last_became_active":"2024-09-12T01:34:07.573810+0000","last_became_peered":"2024-09-12T01:34:07.573810+0000","last_unstale":"2024-09-12T01:34:07.576448+0000","last_undegraded":"2024-09-12T01:34:07.576448+0000","last_fullsized":"2024-09-12T01:34:07.576448+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:56.562905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.577590+0000","last_change":"2024-09-12T01:34:07.572918+0000","last_active":"2024-09-12T01:34:07.577590+0000","last_peered":"2024-09-12T01:34:07.577590+0000","last_clean":"2024-09-12T01:34:07.577590+0000","last_became_active":"2024-09-12T01:34:07.572649+0000","last_became_peered":"2024-09-12T01:34:07.572649+0000","last_unstale":"2024-09-12T01:34:07.577590+0000","last_undegraded":"2024-09-12T01:34:07.577590+0000","last_fullsized":"2024-09-12T01:34:07.577590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:37.323845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.580009+0000","last_change":"2024-09-12T01:34:07.571121+0000","last_active":"2024-09-12T01:34:07.580009+0000","last_peered":"2024-09-12T01:34:07.580009+0000","last_clean":"2024-09-12T01:34:07.580009+0000","last_became_active":"2024-09-12T01:34:07.570676+0000","last_became_peered":"2024-09-12T01:34:07.570676+0000","last_unstale":"2024-09-12T01:34:07.580009+0000","last_undegraded":"2024-09-12T01:34:07.580009+0000","last_fullsized":"2024-09-12T01:34:07.580009+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:19.574669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567753+0000","last_change":"2024-09-12T01:34:07.581851+0000","last_active":"2024-09-12T01:34:08.567753+0000","last_peered":"2024-09-12T01:34:08.567753+0000","last_clean":"2024-09-12T01:34:08.567753+0000","last_became_active":"2024-09-12T01:34:07.581562+0000","last_became_peered":"2024-09-12T01:34:07.581562+0000","last_unstale":"2024-09-12T01:34:08.567753+0000","last_undegraded":"2024-09-12T01:34:08.567753+0000","last_fullsized":"2024-09-12T01:34:08.567753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.266349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.559288+0000","last_change":"2024-09-12T01:34:06.559288+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.559288+0000","last_undegraded":"2024-09-12T01:34:06.559288+0000","last_fullsized":"2024-09-12T01:34:06.559288+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.556833+0000","last_change":"2024-09-12T01:34:06.556833+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.556833+0000","last_undegraded":"2024-09-12T01:34:06.556833+0000","last_fullsized":"2024-09-12T01:34:06.556833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.574203+0000","last_change":"2024-09-12T01:34:07.570064+0000","last_active":"2024-09-12T01:34:07.574203+0000","last_peered":"2024-09-12T01:34:07.574203+0000","last_clean":"2024-09-12T01:34:07.574203+0000","last_became_active":"2024-09-12T01:34:07.569772+0000","last_became_peered":"2024-09-12T01:34:07.569772+0000","last_unstale":"2024-09-12T01:34:07.574203+0000","last_undegraded":"2024-09-12T01:34:07.574203+0000","last_fullsized":"2024-09-12T01:34:07.574203+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:21:38.197322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.574111+0000","last_change":"2024-09-12T01:34:07.570352+0000","last_active":"2024-09-12T01:34:07.574111+0000","last_peered":"2024-09-12T01:34:07.574111+0000","last_clean":"2024-09-12T01:34:07.574111+0000","last_became_active":"2024-09-12T01:34:07.569630+0000","last_became_peered":"2024-09-12T01:34:07.569630+0000","last_unstale":"2024-09-12T01:34:07.574111+0000","last_undegraded":"2024-09-12T01:34:07.574111+0000","last_fullsized":"2024-09-12T01:34:07.574111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:40.751596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584594+0000","last_change":"2024-09-12T01:34:07.583580+0000","last_active":"2024-09-12T01:34:07.584594+0000","last_peered":"2024-09-12T01:34:07.584594+0000","last_clean":"2024-09-12T01:34:07.584594+0000","last_became_active":"2024-09-12T01:34:07.580853+0000","last_became_peered":"2024-09-12T01:34:07.580853+0000","last_unstale":"2024-09-12T01:34:07.584594+0000","last_undegraded":"2024-09-12T01:34:07.584594+0000","last_fullsized":"2024-09-12T01:34:07.584594+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:21.696603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584661+0000","last_change":"2024-09-12T01:34:07.580355+0000","last_active":"2024-09-12T01:34:07.584661+0000","last_peered":"2024-09-12T01:34:07.584661+0000","last_clean":"2024-09-12T01:34:07.584661+0000","last_became_active":"2024-09-12T01:34:07.580020+0000","last_became_peered":"2024-09-12T01:34:07.580020+0000","last_unstale":"2024-09-12T01:34:07.584661+0000","last_undegraded":"2024-09-12T01:34:07.584661+0000","last_fullsized":"2024-09-12T01:34:07.584661+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:22.777538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584013+0000","last_change":"2024-09-12T01:34:07.581833+0000","last_active":"2024-09-12T01:34:07.584013+0000","last_peered":"2024-09-12T01:34:07.584013+0000","last_clean":"2024-09-12T01:34:07.584013+0000","last_became_active":"2024-09-12T01:34:07.581526+0000","last_became_peered":"2024-09-12T01:34:07.581526+0000","last_unstale":"2024-09-12T01:34:07.584013+0000","last_undegraded":"2024-09-12T01:34:07.584013+0000","last_fullsized":"2024-09-12T01:34:07.584013+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:28.448686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-12T01:34:06.573376+0000","last_change":"2024-09-12T01:34:06.572433+0000","last_active":"2024-09-12T01:34:06.573376+0000","last_peered":"2024-09-12T01:34:06.573376+0000","last_clean":"2024-09-12T01:34:06.573376+0000","last_became_active":"2024-09-12T01:34:06.572147+0000","last_became_peered":"2024-09-12T01:34:06.572147+0000","last_unstale":"2024-09-12T01:34:06.573376+0000","last_undegraded":"2024-09-12T01:34:06.573376+0000","last_fullsized":"2024-09-12T01:34:06.573376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:32.352391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.581274+0000","last_change":"2024-09-12T01:34:06.573185+0000","last_active":"2024-09-12T01:34:07.581274+0000","last_peered":"2024-09-12T01:34:07.581274+0000","last_clean":"2024-09-12T01:34:07.581274+0000","last_became_active":"2024-09-12T01:34:06.572814+0000","last_became_peered":"2024-09-12T01:34:06.572814+0000","last_unstale":"2024-09-12T01:34:07.581274+0000","last_undegraded":"2024-09-12T01:34:07.581274+0000","last_fullsized":"2024-09-12T01:34:07.581274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:35.191411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568251+0000","last_change":"2024-09-12T01:34:07.575202+0000","last_active":"2024-09-12T01:34:08.568251+0000","last_peered":"2024-09-12T01:34:08.568251+0000","last_clean":"2024-09-12T01:34:08.568251+0000","last_became_active":"2024-09-12T01:34:07.574888+0000","last_became_peered":"2024-09-12T01:34:07.574888+0000","last_unstale":"2024-09-12T01:34:08.568251+0000","last_undegraded":"2024-09-12T01:34:08.568251+0000","last_fullsized":"2024-09-12T01:34:08.568251+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:17.171969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.558243+0000","last_change":"2024-09-12T01:34:06.558243+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.558243+0000","last_undegraded":"2024-09-12T01:34:06.558243+0000","last_fullsized":"2024-09-12T01:34:06.558243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562765+0000","last_change":"2024-09-12T01:34:06.571529+0000","last_active":"2024-09-12T01:34:07.562765+0000","last_peered":"2024-09-12T01:34:07.562765+0000","last_clean":"2024-09-12T01:34:07.562765+0000","last_became_active":"2024-09-12T01:34:06.571192+0000","last_became_peered":"2024-09-12T01:34:06.571192+0000","last_unstale":"2024-09-12T01:34:07.562765+0000","last_undegraded":"2024-09-12T01:34:07.562765+0000","last_fullsized":"2024-09-12T01:34:07.562765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:18.183678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568020+0000","last_change":"2024-09-12T01:34:07.572915+0000","last_active":"2024-09-12T01:34:08.568020+0000","last_peered":"2024-09-12T01:34:08.568020+0000","last_clean":"2024-09-12T01:34:08.568020+0000","last_became_active":"2024-09-12T01:34:07.572585+0000","last_became_peered":"2024-09-12T01:34:07.572585+0000","last_unstale":"2024-09-12T01:34:08.568020+0000","last_undegraded":"2024-09-12T01:34:08.568020+0000","last_fullsized":"2024-09-12T01:34:08.568020+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:31.058724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.576425+0000","last_change":"2024-09-12T01:34:07.573978+0000","last_active":"2024-09-12T01:34:07.576425+0000","last_peered":"2024-09-12T01:34:07.576425+0000","last_clean":"2024-09-12T01:34:07.576425+0000","last_became_active":"2024-09-12T01:34:07.573412+0000","last_became_peered":"2024-09-12T01:34:07.573412+0000","last_unstale":"2024-09-12T01:34:07.576425+0000","last_undegraded":"2024-09-12T01:34:07.576425+0000","last_fullsized":"2024-09-12T01:34:07.576425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:05.655767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568445+0000","last_change":"2024-09-12T01:34:06.566440+0000","last_active":"2024-09-12T01:34:08.568445+0000","last_peered":"2024-09-12T01:34:08.568445+0000","last_clean":"2024-09-12T01:34:08.568445+0000","last_became_active":"2024-09-12T01:34:06.566046+0000","last_became_peered":"2024-09-12T01:34:06.566046+0000","last_unstale":"2024-09-12T01:34:08.568445+0000","last_undegraded":"2024-09-12T01:34:08.568445+0000","last_fullsized":"2024-09-12T01:34:08.568445+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:27.443857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557220+0000","last_change":"2024-09-12T01:34:06.557220+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557220+0000","last_undegraded":"2024-09-12T01:34:06.557220+0000","last_fullsized":"2024-09-12T01:34:06.557220+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568357+0000","last_change":"2024-09-12T01:34:06.576115+0000","last_active":"2024-09-12T01:34:08.568357+0000","last_peered":"2024-09-12T01:34:08.568357+0000","last_clean":"2024-09-12T01:34:08.568357+0000","last_became_active":"2024-09-12T01:34:06.575873+0000","last_became_peered":"2024-09-12T01:34:06.575873+0000","last_unstale":"2024-09-12T01:34:08.568357+0000","last_undegraded":"2024-09-12T01:34:08.568357+0000","last_fullsized":"2024-09-12T01:34:08.568357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.691775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557835+0000","last_change":"2024-09-12T01:34:06.557835+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557835+0000","last_undegraded":"2024-09-12T01:34:06.557835+0000","last_fullsized":"2024-09-12T01:34:06.557835+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568240+0000","last_change":"2024-09-12T01:34:06.566516+0000","last_active":"2024-09-12T01:34:08.568240+0000","last_peered":"2024-09-12T01:34:08.568240+0000","last_clean":"2024-09-12T01:34:08.568240+0000","last_became_active":"2024-09-12T01:34:06.566186+0000","last_became_peered":"2024-09-12T01:34:06.566186+0000","last_unstale":"2024-09-12T01:34:08.568240+0000","last_undegraded":"2024-09-12T01:34:08.568240+0000","last_fullsized":"2024-09-12T01:34:08.568240+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:59:30.145684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567848+0000","last_change":"2024-09-12T01:34:07.567627+0000","last_active":"2024-09-12T01:34:08.567848+0000","last_peered":"2024-09-12T01:34:08.567848+0000","last_clean":"2024-09-12T01:34:08.567848+0000","last_became_active":"2024-09-12T01:34:07.567381+0000","last_became_peered":"2024-09-12T01:34:07.567381+0000","last_unstale":"2024-09-12T01:34:08.567848+0000","last_undegraded":"2024-09-12T01:34:08.567848+0000","last_fullsized":"2024-09-12T01:34:08.567848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:30.783361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562642+0000","last_change":"2024-09-12T01:34:06.573250+0000","last_active":"2024-09-12T01:34:07.562642+0000","last_peered":"2024-09-12T01:34:07.562642+0000","last_clean":"2024-09-12T01:34:07.562642+0000","last_became_active":"2024-09-12T01:34:06.572929+0000","last_became_peered":"2024-09-12T01:34:06.572929+0000","last_unstale":"2024-09-12T01:34:07.562642+0000","last_undegraded":"2024-09-12T01:34:07.562642+0000","last_fullsized":"2024-09-12T01:34:07.562642+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:43.363051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568123+0000","last_change":"2024-09-12T01:34:07.566974+0000","last_active":"2024-09-12T01:34:08.568123+0000","last_peered":"2024-09-12T01:34:08.568123+0000","last_clean":"2024-09-12T01:34:08.568123+0000","last_became_active":"2024-09-12T01:34:07.566688+0000","last_became_peered":"2024-09-12T01:34:07.566688+0000","last_unstale":"2024-09-12T01:34:08.568123+0000","last_undegraded":"2024-09-12T01:34:08.568123+0000","last_fullsized":"2024-09-12T01:34:08.568123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:35.009027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568341+0000","last_change":"2024-09-12T01:34:07.576753+0000","last_active":"2024-09-12T01:34:08.568341+0000","last_peered":"2024-09-12T01:34:08.568341+0000","last_clean":"2024-09-12T01:34:08.568341+0000","last_became_active":"2024-09-12T01:34:07.576461+0000","last_became_peered":"2024-09-12T01:34:07.576461+0000","last_unstale":"2024-09-12T01:34:08.568341+0000","last_undegraded":"2024-09-12T01:34:08.568341+0000","last_fullsized":"2024-09-12T01:34:08.568341+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:12.831326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.589168+0000","last_change":"2024-09-12T01:34:06.572508+0000","last_active":"2024-09-12T01:34:07.589168+0000","last_peered":"2024-09-12T01:34:07.589168+0000","last_clean":"2024-09-12T01:34:07.589168+0000","last_became_active":"2024-09-12T01:34:06.572222+0000","last_became_peered":"2024-09-12T01:34:06.572222+0000","last_unstale":"2024-09-12T01:34:07.589168+0000","last_undegraded":"2024-09-12T01:34:07.589168+0000","last_fullsized":"2024-09-12T01:34:07.589168+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:12.959609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.574094+0000","last_change":"2024-09-12T01:34:07.571161+0000","last_active":"2024-09-12T01:34:07.574094+0000","last_peered":"2024-09-12T01:34:07.574094+0000","last_clean":"2024-09-12T01:34:07.574094+0000","last_became_active":"2024-09-12T01:34:07.570891+0000","last_became_peered":"2024-09-12T01:34:07.570891+0000","last_unstale":"2024-09-12T01:34:07.574094+0000","last_undegraded":"2024-09-12T01:34:07.574094+0000","last_fullsized":"2024-09-12T01:34:07.574094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:02.383752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562901+0000","last_change":"2024-09-12T01:34:06.574086+0000","last_active":"2024-09-12T01:34:07.562901+0000","last_peered":"2024-09-12T01:34:07.562901+0000","last_clean":"2024-09-12T01:34:07.562901+0000","last_became_active":"2024-09-12T01:34:06.573682+0000","last_became_peered":"2024-09-12T01:34:06.573682+0000","last_unstale":"2024-09-12T01:34:07.562901+0000","last_undegraded":"2024-09-12T01:34:07.562901+0000","last_fullsized":"2024-09-12T01:34:07.562901+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:38.064258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584434+0000","last_change":"2024-09-12T01:34:06.563081+0000","last_active":"2024-09-12T01:34:07.584434+0000","last_peered":"2024-09-12T01:34:07.584434+0000","last_clean":"2024-09-12T01:34:07.584434+0000","last_became_active":"2024-09-12T01:34:06.562728+0000","last_became_peered":"2024-09-12T01:34:06.562728+0000","last_unstale":"2024-09-12T01:34:07.584434+0000","last_undegraded":"2024-09-12T01:34:07.584434+0000","last_fullsized":"2024-09-12T01:34:07.584434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:56.551827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.558269+0000","last_change":"2024-09-12T01:34:06.558269+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.558269+0000","last_undegraded":"2024-09-12T01:34:06.558269+0000","last_fullsized":"2024-09-12T01:34:06.558269+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.560878+0000","last_change":"2024-09-12T01:34:06.573266+0000","last_active":"2024-09-12T01:34:07.560878+0000","last_peered":"2024-09-12T01:34:07.560878+0000","last_clean":"2024-09-12T01:34:07.560878+0000","last_became_active":"2024-09-12T01:34:06.572679+0000","last_became_peered":"2024-09-12T01:34:06.572679+0000","last_unstale":"2024-09-12T01:34:07.560878+0000","last_undegraded":"2024-09-12T01:34:07.560878+0000","last_fullsized":"2024-09-12T01:34:07.560878+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:50.930775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568150+0000","last_change":"2024-09-12T01:34:07.572416+0000","last_active":"2024-09-12T01:34:08.568150+0000","last_peered":"2024-09-12T01:34:08.568150+0000","last_clean":"2024-09-12T01:34:08.568150+0000","last_became_active":"2024-09-12T01:34:07.572092+0000","last_became_peered":"2024-09-12T01:34:07.572092+0000","last_unstale":"2024-09-12T01:34:08.568150+0000","last_undegraded":"2024-09-12T01:34:08.568150+0000","last_fullsized":"2024-09-12T01:34:08.568150+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:55.078476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.560460+0000","last_change":"2024-09-12T01:34:06.562339+0000","last_active":"2024-09-12T01:34:07.560460+0000","last_peered":"2024-09-12T01:34:07.560460+0000","last_clean":"2024-09-12T01:34:07.560460+0000","last_became_active":"2024-09-12T01:34:06.561990+0000","last_became_peered":"2024-09-12T01:34:06.561990+0000","last_unstale":"2024-09-12T01:34:07.560460+0000","last_undegraded":"2024-09-12T01:34:07.560460+0000","last_fullsized":"2024-09-12T01:34:07.560460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:09.175440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.583991+0000","last_change":"2024-09-12T01:34:07.581576+0000","last_active":"2024-09-12T01:34:07.583991+0000","last_peered":"2024-09-12T01:34:07.583991+0000","last_clean":"2024-09-12T01:34:07.583991+0000","last_became_active":"2024-09-12T01:34:07.580839+0000","last_became_peered":"2024-09-12T01:34:07.580839+0000","last_unstale":"2024-09-12T01:34:07.583991+0000","last_undegraded":"2024-09-12T01:34:07.583991+0000","last_fullsized":"2024-09-12T01:34:07.583991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:32.726522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.561435+0000","last_change":"2024-09-12T01:34:06.577637+0000","last_active":"2024-09-12T01:34:07.561435+0000","last_peered":"2024-09-12T01:34:07.561435+0000","last_clean":"2024-09-12T01:34:07.561435+0000","last_became_active":"2024-09-12T01:34:06.577338+0000","last_became_peered":"2024-09-12T01:34:06.577338+0000","last_unstale":"2024-09-12T01:34:07.561435+0000","last_undegraded":"2024-09-12T01:34:07.561435+0000","last_fullsized":"2024-09-12T01:34:07.561435+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:44.282477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567958+0000","last_change":"2024-09-12T01:34:07.563962+0000","last_active":"2024-09-12T01:34:08.567958+0000","last_peered":"2024-09-12T01:34:08.567958+0000","last_clean":"2024-09-12T01:34:08.567958+0000","last_became_active":"2024-09-12T01:34:07.563649+0000","last_became_peered":"2024-09-12T01:34:07.563649+0000","last_unstale":"2024-09-12T01:34:08.567958+0000","last_undegraded":"2024-09-12T01:34:08.567958+0000","last_fullsized":"2024-09-12T01:34:08.567958+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:36:19.605889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-12T01:34:06.566559+0000","last_change":"2024-09-12T01:34:06.564220+0000","last_active":"2024-09-12T01:34:06.566559+0000","last_peered":"2024-09-12T01:34:06.566559+0000","last_clean":"2024-09-12T01:34:06.566559+0000","last_became_active":"2024-09-12T01:34:06.563912+0000","last_became_peered":"2024-09-12T01:34:06.563912+0000","last_unstale":"2024-09-12T01:34:06.566559+0000","last_undegraded":"2024-09-12T01:34:06.566559+0000","last_fullsized":"2024-09-12T01:34:06.566559+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:26.470501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-12T01:34:06.557400+0000","last_change":"2024-09-12T01:34:06.557400+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.557400+0000","last_undegraded":"2024-09-12T01:34:06.557400+0000","last_fullsized":"2024-09-12T01:34:06.557400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-12T01:34:06.569286+0000","last_change":"2024-09-12T01:34:06.567684+0000","last_active":"2024-09-12T01:34:06.569286+0000","last_peered":"2024-09-12T01:34:06.569286+0000","last_clean":"2024-09-12T01:34:06.569286+0000","last_became_active":"2024-09-12T01:34:06.567184+0000","last_became_peered":"2024-09-12T01:34:06.567184+0000","last_unstale":"2024-09-12T01:34:06.569286+0000","last_undegraded":"2024-09-12T01:34:06.569286+0000","last_fullsized":"2024-09-12T01:34:06.569286+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:23.197060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.580895+0000","last_change":"2024-09-12T01:34:07.573865+0000","last_active":"2024-09-12T01:34:07.580895+0000","last_peered":"2024-09-12T01:34:07.580895+0000","last_clean":"2024-09-12T01:34:07.580895+0000","last_became_active":"2024-09-12T01:34:07.573578+0000","last_became_peered":"2024-09-12T01:34:07.573578+0000","last_unstale":"2024-09-12T01:34:07.580895+0000","last_undegraded":"2024-09-12T01:34:07.580895+0000","last_fullsized":"2024-09-12T01:34:07.580895+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:03.055098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567833+0000","last_change":"2024-09-12T01:34:06.566830+0000","last_active":"2024-09-12T01:34:08.567833+0000","last_peered":"2024-09-12T01:34:08.567833+0000","last_clean":"2024-09-12T01:34:08.567833+0000","last_became_active":"2024-09-12T01:34:06.566321+0000","last_became_peered":"2024-09-12T01:34:06.566321+0000","last_unstale":"2024-09-12T01:34:08.567833+0000","last_undegraded":"2024-09-12T01:34:08.567833+0000","last_fullsized":"2024-09-12T01:34:08.567833+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:31.718104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:06.545580+0000","last_change":"2024-09-12T01:34:06.545580+0000","last_active":"2024-09-12T01:34:06.545580+0000","last_peered":"2024-09-12T01:34:06.545580+0000","last_clean":"2024-09-12T01:34:06.545580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:06.545580+0000","last_undegraded":"2024-09-12T01:34:06.545580+0000","last_fullsized":"2024-09-12T01:34:06.545580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T01:34:05.540751+0000","last_change":"2024-09-12T01:34:05.540751+0000","last_active":"2024-09-12T01:34:05.540751+0000","last_peered":"2024-09-12T01:34:05.540751+0000","last_clean":"2024-09-12T01:34:05.540751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T01:34:05.540751+0000","last_undegraded":"2024-09-12T01:34:05.540751+0000","last_fullsized":"2024-09-12T01:34:05.540751+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-12T01:34:02.259557+0000","last_change":"2024-09-12T01:33:01.096542+0000","last_active":"2024-09-12T01:34:02.259557+0000","last_peered":"2024-09-12T01:34:02.259557+0000","last_clean":"2024-09-12T01:34:02.259557+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:34:02.259557+0000","last_undegraded":"2024-09-12T01:34:02.259557+0000","last_fullsized":"2024-09-12T01:34:02.259557+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_clean_scrub_stamp":"2024-09-12T01:32:58.512553+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:48:58.741813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":60,"acting":60,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1716,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":32,"acting":32,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":999424,"data_stored":763507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":173139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":173139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":173139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":471040,"data_stored":201520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":446464,"data_stored":193272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":446464,"data_stored":193252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1024000,"data_stored":776383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:34:12.542 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-09-12T01:34:12.542+0000 7f897292b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T01:34:12.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:34:12.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:34:15.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:34:15.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.537+0000 7f58d47b9640 1 Processor -- start 2024-09-12T01:34:15.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.537+0000 7f58d47b9640 1 -- start start 2024-09-12T01:34:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.538+0000 7f58d47b9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58cc15e8d0 0x7f58cc06dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.538+0000 7f58d47b9640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58cc0cb580 0x7f58cc073720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.538+0000 7f58d47b9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58cc06f530 con 0x7f58cc0acdd0 2024-09-12T01:34:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.538+0000 7f58d47b9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f58cc0cbbb0 con 0x7f58cc15e8d0 2024-09-12T01:34:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.538+0000 7f58d47b9640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f58cc0cbe90 con 0x7f58cc0cb580 2024-09-12T01:34:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.539+0000 7f58d2555640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58cc15e8d0 0x7f58cc06dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.539+0000 7f58d1d54640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f58cc0acdd0 0x7f58cc0cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55116/0 (socket says 172.21.15.16:55116) 2024-09-12T01:34:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.539+0000 7f58d1d54640 1 -- 172.21.15.16:0/92281213 learned_addr learned my addr 172.21.15.16:0/92281213 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.539+0000 7f58d2d56640 1 --2- 172.21.15.16:0/92281213 >> v2:172.21.15.33:3300/0 conn(0x7f58cc0cb580 0x7f58cc073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.539+0000 7f58d2555640 1 -- 172.21.15.16:0/92281213 >> v2:172.21.15.33:3300/0 conn(0x7f58cc0cb580 msgr2=0x7f58cc073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.539+0000 7f58d2555640 1 --2- 172.21.15.16:0/92281213 >> v2:172.21.15.33:3300/0 conn(0x7f58cc0cb580 0x7f58cc073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.539+0000 7f58d2555640 1 -- 172.21.15.16:0/92281213 >> v1:172.21.15.16:6789/0 conn(0x7f58cc0acdd0 legacy=0x7f58cc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.539+0000 7f58d1553640 1 -- 172.21.15.16:0/92281213 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2338078853 0 0) 0x7f58cc06f530 con 0x7f58cc0acdd0 2024-09-12T01:34:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.539+0000 7f58d2555640 1 -- 172.21.15.16:0/92281213 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc0ca5f0 con 0x7f58cc15e8d0 2024-09-12T01:34:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.540+0000 7f58d2555640 1 --2- 172.21.15.16:0/92281213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc15e8d0 0x7f58cc06dd90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f58c80097a0 tx=0x7f58c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eaf6bc302db46ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.540+0000 7f58d1553640 1 -- 172.21.15.16:0/92281213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8037070 con 0x7f58cc15e8d0 2024-09-12T01:34:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.540+0000 7f58d1553640 1 -- 172.21.15.16:0/92281213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c8032440 con 0x7f58cc15e8d0 2024-09-12T01:34:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.540+0000 7f58d1553640 1 -- 172.21.15.16:0/92281213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8032810 con 0x7f58cc15e8d0 2024-09-12T01:34:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.540+0000 7f58d47b9640 1 -- 172.21.15.16:0/92281213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc15e8d0 msgr2=0x7f58cc06dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.540+0000 7f58d47b9640 1 --2- 172.21.15.16:0/92281213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc15e8d0 0x7f58cc06dd90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f58c80097a0 tx=0x7f58c802a1e0 comp rx=0 tx=0).stop 2024-09-12T01:34:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.541+0000 7f58d47b9640 1 -- 172.21.15.16:0/92281213 shutdown_connections 2024-09-12T01:34:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.541+0000 7f58d47b9640 1 --2- 172.21.15.16:0/92281213 >> v2:172.21.15.33:3300/0 conn(0x7f58cc0cb580 0x7f58cc073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.541+0000 7f58d47b9640 1 --2- 172.21.15.16:0/92281213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc15e8d0 0x7f58cc06dd90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.541+0000 7f58d47b9640 1 -- 172.21.15.16:0/92281213 >> 172.21.15.16:0/92281213 conn(0x7f58cc0cffa0 msgr2=0x7f58cc0d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.541+0000 7f58d47b9640 1 -- 172.21.15.16:0/92281213 shutdown_connections 2024-09-12T01:34:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.541+0000 7f58d47b9640 1 -- 172.21.15.16:0/92281213 wait complete. 2024-09-12T01:34:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.542+0000 7f58d47b9640 1 Processor -- start 2024-09-12T01:34:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.542+0000 7f58d47b9640 1 -- start start 2024-09-12T01:34:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d47b9640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc0acdd0 0x7f58cc14e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d47b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0cb580 0x7f58cc14e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d47b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc15e8d0 0x7f58cc09cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d47b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc0ce860 con 0x7f58cc0cb580 2024-09-12T01:34:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d47b9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc0cabb0 con 0x7f58cc15e8d0 2024-09-12T01:34:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d47b9640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc0ce9e0 con 0x7f58cc0acdd0 2024-09-12T01:34:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d1d54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0cb580 0x7f58cc14e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d1d54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0cb580 0x7f58cc14e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41478/0 (socket says 172.21.15.16:41478) 2024-09-12T01:34:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d1d54640 1 -- 172.21.15.16:0/3408876002 learned_addr learned my addr 172.21.15.16:0/3408876002 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d2555640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc0acdd0 0x7f58cc14e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d2d56640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc15e8d0 0x7f58cc09cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d1d54640 1 -- 172.21.15.16:0/3408876002 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc0acdd0 msgr2=0x7f58cc14e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d1d54640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc0acdd0 0x7f58cc14e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d1d54640 1 -- 172.21.15.16:0/3408876002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc15e8d0 msgr2=0x7f58cc09cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d1d54640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc15e8d0 0x7f58cc09cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d1d54640 1 -- 172.21.15.16:0/3408876002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc09d890 con 0x7f58cc0cb580 2024-09-12T01:34:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d2d56640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc15e8d0 0x7f58cc09cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58d1d54640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0cb580 0x7f58cc14e060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00c4d0 tx=0x7f58bc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58c2ffd640 1 -- 172.21.15.16:0/3408876002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc018070 con 0x7f58cc0cb580 2024-09-12T01:34:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58c2ffd640 1 -- 172.21.15.16:0/3408876002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58bc007500 con 0x7f58cc0cb580 2024-09-12T01:34:15.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.543+0000 7f58c2ffd640 1 -- 172.21.15.16:0/3408876002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc004a20 con 0x7f58cc0cb580 2024-09-12T01:34:15.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.544+0000 7f58d47b9640 1 -- 172.21.15.16:0/3408876002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc14ee60 con 0x7f58cc0cb580 2024-09-12T01:34:15.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.544+0000 7f58d47b9640 1 -- 172.21.15.16:0/3408876002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc14f420 con 0x7f58cc0cb580 2024-09-12T01:34:15.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.545+0000 7f58d47b9640 1 -- 172.21.15.16:0/3408876002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58cc06dd90 con 0x7f58cc0cb580 2024-09-12T01:34:15.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.545+0000 7f58c2ffd640 1 -- 172.21.15.16:0/3408876002 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58bc004bc0 con 0x7f58cc0cb580 2024-09-12T01:34:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.546+0000 7f58c2ffd640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f58b006be60 0x7f58b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.546+0000 7f58c2ffd640 1 -- 172.21.15.16:0/3408876002 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f58bc08a220 con 0x7f58cc0cb580 2024-09-12T01:34:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.546+0000 7f58d2555640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f58b006be60 0x7f58b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.546+0000 7f58d2555640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f58b006be60 0x7f58b006e320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f58c8034000 tx=0x7f58c802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:15.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.550+0000 7f58c2ffd640 1 -- 172.21.15.16:0/3408876002 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58cc09d890 con 0x7f58cc0cb580 2024-09-12T01:34:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.726+0000 7f58d47b9640 1 -- 172.21.15.16:0/3408876002 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58cc0cf850 con 0x7f58b006be60 2024-09-12T01:34:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.732+0000 7f58c2ffd640 1 -- 172.21.15.16:0/3408876002 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f58cc0cf850 con 0x7f58b006be60 2024-09-12T01:34:15.734 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:15.735 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:34:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.735+0000 7f58d47b9640 1 -- 172.21.15.16:0/3408876002 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f58b006be60 msgr2=0x7f58b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.735+0000 7f58d47b9640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f58b006be60 0x7f58b006e320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f58c8034000 tx=0x7f58c802e000 comp rx=0 tx=0).stop 2024-09-12T01:34:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.735+0000 7f58d47b9640 1 -- 172.21.15.16:0/3408876002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0cb580 msgr2=0x7f58cc14e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.735+0000 7f58d47b9640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0cb580 0x7f58cc14e060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00c4d0 tx=0x7f58bc00c990 comp rx=0 tx=0).stop 2024-09-12T01:34:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.735+0000 7f58d47b9640 1 -- 172.21.15.16:0/3408876002 shutdown_connections 2024-09-12T01:34:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.735+0000 7f58d47b9640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f58b006be60 0x7f58b006e320 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.735+0000 7f58d47b9640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58cc15e8d0 0x7f58cc09cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.735+0000 7f58d47b9640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0cb580 0x7f58cc14e060 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.735+0000 7f58d47b9640 1 --2- 172.21.15.16:0/3408876002 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f58cc0acdd0 0x7f58cc14e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:15.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.735+0000 7f58d47b9640 1 -- 172.21.15.16:0/3408876002 >> 172.21.15.16:0/3408876002 conn(0x7f58cc0cffa0 msgr2=0x7f58cc06e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:15.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.735+0000 7f58d47b9640 1 -- 172.21.15.16:0/3408876002 shutdown_connections 2024-09-12T01:34:15.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.735+0000 7f58d47b9640 1 -- 172.21.15.16:0/3408876002 wait complete. 2024-09-12T01:34:15.745 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-12T01:34:14.483689+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237616,"kb_used_data":4728,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737104,"statfs":{"total":773094113280,"available":772850794496,"internally_reserved":0,"allocated":4841472,"data_stored":2757780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.932569"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568397+0000","last_change":"2024-09-12T01:34:07.568217+0000","last_active":"2024-09-12T01:34:08.568397+0000","last_peered":"2024-09-12T01:34:08.568397+0000","last_clean":"2024-09-12T01:34:08.568397+0000","last_became_active":"2024-09-12T01:34:07.567979+0000","last_became_peered":"2024-09-12T01:34:07.567979+0000","last_unstale":"2024-09-12T01:34:08.568397+0000","last_undegraded":"2024-09-12T01:34:08.568397+0000","last_fullsized":"2024-09-12T01:34:08.568397+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:07.721403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567960+0000","last_change":"2024-09-12T01:34:07.566615+0000","last_active":"2024-09-12T01:34:08.567960+0000","last_peered":"2024-09-12T01:34:08.567960+0000","last_clean":"2024-09-12T01:34:08.567960+0000","last_became_active":"2024-09-12T01:34:07.566309+0000","last_became_peered":"2024-09-12T01:34:07.566309+0000","last_unstale":"2024-09-12T01:34:08.567960+0000","last_undegraded":"2024-09-12T01:34:08.567960+0000","last_fullsized":"2024-09-12T01:34:08.567960+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:10.428484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.009800+0000","last_change":"2024-09-12T01:34:07.580683+0000","last_active":"2024-09-12T01:34:09.009800+0000","last_peered":"2024-09-12T01:34:09.009800+0000","last_clean":"2024-09-12T01:34:09.009800+0000","last_became_active":"2024-09-12T01:34:07.580385+0000","last_became_peered":"2024-09-12T01:34:07.580385+0000","last_unstale":"2024-09-12T01:34:09.009800+0000","last_undegraded":"2024-09-12T01:34:09.009800+0000","last_fullsized":"2024-09-12T01:34:09.009800+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:01.704612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.659402+0000","last_change":"2024-09-12T01:34:07.574151+0000","last_active":"2024-09-12T01:34:08.659402+0000","last_peered":"2024-09-12T01:34:08.659402+0000","last_clean":"2024-09-12T01:34:08.659402+0000","last_became_active":"2024-09-12T01:34:07.573810+0000","last_became_peered":"2024-09-12T01:34:07.573810+0000","last_unstale":"2024-09-12T01:34:08.659402+0000","last_undegraded":"2024-09-12T01:34:08.659402+0000","last_fullsized":"2024-09-12T01:34:08.659402+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:26:03.103089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.659291+0000","last_change":"2024-09-12T01:34:07.572918+0000","last_active":"2024-09-12T01:34:08.659291+0000","last_peered":"2024-09-12T01:34:08.659291+0000","last_clean":"2024-09-12T01:34:08.659291+0000","last_became_active":"2024-09-12T01:34:07.572649+0000","last_became_peered":"2024-09-12T01:34:07.572649+0000","last_unstale":"2024-09-12T01:34:08.659291+0000","last_undegraded":"2024-09-12T01:34:08.659291+0000","last_fullsized":"2024-09-12T01:34:08.659291+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:47.713479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.580009+0000","last_change":"2024-09-12T01:34:07.571121+0000","last_active":"2024-09-12T01:34:07.580009+0000","last_peered":"2024-09-12T01:34:07.580009+0000","last_clean":"2024-09-12T01:34:07.580009+0000","last_became_active":"2024-09-12T01:34:07.570676+0000","last_became_peered":"2024-09-12T01:34:07.570676+0000","last_unstale":"2024-09-12T01:34:07.580009+0000","last_undegraded":"2024-09-12T01:34:07.580009+0000","last_fullsized":"2024-09-12T01:34:07.580009+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:19.574669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584921+0000","last_change":"2024-09-12T01:34:07.581338+0000","last_active":"2024-09-12T01:34:07.584921+0000","last_peered":"2024-09-12T01:34:07.584921+0000","last_clean":"2024-09-12T01:34:07.584921+0000","last_became_active":"2024-09-12T01:34:07.581049+0000","last_became_peered":"2024-09-12T01:34:07.581049+0000","last_unstale":"2024-09-12T01:34:07.584921+0000","last_undegraded":"2024-09-12T01:34:07.584921+0000","last_fullsized":"2024-09-12T01:34:07.584921+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:21.696603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567753+0000","last_change":"2024-09-12T01:34:07.581851+0000","last_active":"2024-09-12T01:34:08.567753+0000","last_peered":"2024-09-12T01:34:08.567753+0000","last_clean":"2024-09-12T01:34:08.567753+0000","last_became_active":"2024-09-12T01:34:07.581562+0000","last_became_peered":"2024-09-12T01:34:07.581562+0000","last_unstale":"2024-09-12T01:34:08.567753+0000","last_undegraded":"2024-09-12T01:34:08.567753+0000","last_fullsized":"2024-09-12T01:34:08.567753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.266349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.010071+0000","last_change":"2024-09-12T01:34:07.588159+0000","last_active":"2024-09-12T01:34:09.010071+0000","last_peered":"2024-09-12T01:34:09.010071+0000","last_clean":"2024-09-12T01:34:09.010071+0000","last_became_active":"2024-09-12T01:34:07.587875+0000","last_became_peered":"2024-09-12T01:34:07.587875+0000","last_unstale":"2024-09-12T01:34:09.010071+0000","last_undegraded":"2024-09-12T01:34:09.010071+0000","last_fullsized":"2024-09-12T01:34:09.010071+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:42.251458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.593815+0000","last_change":"2024-09-12T01:34:07.589889+0000","last_active":"2024-09-12T01:34:07.593815+0000","last_peered":"2024-09-12T01:34:07.593815+0000","last_clean":"2024-09-12T01:34:07.593815+0000","last_became_active":"2024-09-12T01:34:07.589077+0000","last_became_peered":"2024-09-12T01:34:07.589077+0000","last_unstale":"2024-09-12T01:34:07.593815+0000","last_undegraded":"2024-09-12T01:34:07.593815+0000","last_fullsized":"2024-09-12T01:34:07.593815+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:25.387081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.593615+0000","last_change":"2024-09-12T01:34:07.585478+0000","last_active":"2024-09-12T01:34:07.593615+0000","last_peered":"2024-09-12T01:34:07.593615+0000","last_clean":"2024-09-12T01:34:07.593615+0000","last_became_active":"2024-09-12T01:34:07.585188+0000","last_became_peered":"2024-09-12T01:34:07.585188+0000","last_unstale":"2024-09-12T01:34:07.593615+0000","last_undegraded":"2024-09-12T01:34:07.593615+0000","last_fullsized":"2024-09-12T01:34:07.593615+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:53.456847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.009821+0000","last_change":"2024-09-12T01:34:07.575074+0000","last_active":"2024-09-12T01:34:09.009821+0000","last_peered":"2024-09-12T01:34:09.009821+0000","last_clean":"2024-09-12T01:34:09.009821+0000","last_became_active":"2024-09-12T01:34:07.574750+0000","last_became_peered":"2024-09-12T01:34:07.574750+0000","last_unstale":"2024-09-12T01:34:09.009821+0000","last_undegraded":"2024-09-12T01:34:09.009821+0000","last_fullsized":"2024-09-12T01:34:09.009821+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:41.182593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.593788+0000","last_change":"2024-09-12T01:34:07.589879+0000","last_active":"2024-09-12T01:34:07.593788+0000","last_peered":"2024-09-12T01:34:07.593788+0000","last_clean":"2024-09-12T01:34:07.593788+0000","last_became_active":"2024-09-12T01:34:07.589087+0000","last_became_peered":"2024-09-12T01:34:07.589087+0000","last_unstale":"2024-09-12T01:34:07.593788+0000","last_undegraded":"2024-09-12T01:34:07.593788+0000","last_fullsized":"2024-09-12T01:34:07.593788+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:29.720685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568644+0000","last_change":"2024-09-12T01:34:07.572939+0000","last_active":"2024-09-12T01:34:08.568644+0000","last_peered":"2024-09-12T01:34:08.568644+0000","last_clean":"2024-09-12T01:34:08.568644+0000","last_became_active":"2024-09-12T01:34:07.571927+0000","last_became_peered":"2024-09-12T01:34:07.571927+0000","last_unstale":"2024-09-12T01:34:08.568644+0000","last_undegraded":"2024-09-12T01:34:08.568644+0000","last_fullsized":"2024-09-12T01:34:08.568644+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:15.555273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584898+0000","last_change":"2024-09-12T01:34:07.580257+0000","last_active":"2024-09-12T01:34:07.584898+0000","last_peered":"2024-09-12T01:34:07.584898+0000","last_clean":"2024-09-12T01:34:07.584898+0000","last_became_active":"2024-09-12T01:34:07.578392+0000","last_became_peered":"2024-09-12T01:34:07.578392+0000","last_unstale":"2024-09-12T01:34:07.584898+0000","last_undegraded":"2024-09-12T01:34:07.584898+0000","last_fullsized":"2024-09-12T01:34:07.584898+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:03.055098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.705556+0000","last_change":"2024-09-12T01:34:07.585731+0000","last_active":"2024-09-12T01:34:09.705556+0000","last_peered":"2024-09-12T01:34:09.705556+0000","last_clean":"2024-09-12T01:34:09.705556+0000","last_became_active":"2024-09-12T01:34:07.585470+0000","last_became_peered":"2024-09-12T01:34:07.585470+0000","last_unstale":"2024-09-12T01:34:09.705556+0000","last_undegraded":"2024-09-12T01:34:09.705556+0000","last_fullsized":"2024-09-12T01:34:09.705556+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:15.555273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.706200+0000","last_change":"2024-09-12T01:34:07.583927+0000","last_active":"2024-09-12T01:34:09.706200+0000","last_peered":"2024-09-12T01:34:09.706200+0000","last_clean":"2024-09-12T01:34:09.706200+0000","last_became_active":"2024-09-12T01:34:07.583013+0000","last_became_peered":"2024-09-12T01:34:07.583013+0000","last_unstale":"2024-09-12T01:34:09.706200+0000","last_undegraded":"2024-09-12T01:34:09.706200+0000","last_fullsized":"2024-09-12T01:34:09.706200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:58.860398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:10.093201+0000","last_change":"2024-09-12T01:34:07.572710+0000","last_active":"2024-09-12T01:34:10.093201+0000","last_peered":"2024-09-12T01:34:10.093201+0000","last_clean":"2024-09-12T01:34:10.093201+0000","last_became_active":"2024-09-12T01:34:07.572438+0000","last_became_peered":"2024-09-12T01:34:07.572438+0000","last_unstale":"2024-09-12T01:34:10.093201+0000","last_undegraded":"2024-09-12T01:34:10.093201+0000","last_fullsized":"2024-09-12T01:34:10.093201+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:21.696603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.659444+0000","last_change":"2024-09-12T01:34:07.570064+0000","last_active":"2024-09-12T01:34:08.659444+0000","last_peered":"2024-09-12T01:34:08.659444+0000","last_clean":"2024-09-12T01:34:08.659444+0000","last_became_active":"2024-09-12T01:34:07.569772+0000","last_became_peered":"2024-09-12T01:34:07.569772+0000","last_unstale":"2024-09-12T01:34:08.659444+0000","last_undegraded":"2024-09-12T01:34:08.659444+0000","last_fullsized":"2024-09-12T01:34:08.659444+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:30.760933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:10.092917+0000","last_change":"2024-09-12T01:34:07.571648+0000","last_active":"2024-09-12T01:34:10.092917+0000","last_peered":"2024-09-12T01:34:10.092917+0000","last_clean":"2024-09-12T01:34:10.092917+0000","last_became_active":"2024-09-12T01:34:07.571277+0000","last_became_peered":"2024-09-12T01:34:07.571277+0000","last_unstale":"2024-09-12T01:34:10.092917+0000","last_undegraded":"2024-09-12T01:34:10.092917+0000","last_fullsized":"2024-09-12T01:34:10.092917+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:22.777538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.587281+0000","last_change":"2024-09-12T01:34:07.582139+0000","last_active":"2024-09-12T01:34:07.587281+0000","last_peered":"2024-09-12T01:34:07.587281+0000","last_clean":"2024-09-12T01:34:07.587281+0000","last_became_active":"2024-09-12T01:34:07.581859+0000","last_became_peered":"2024-09-12T01:34:07.581859+0000","last_unstale":"2024-09-12T01:34:07.587281+0000","last_undegraded":"2024-09-12T01:34:07.587281+0000","last_fullsized":"2024-09-12T01:34:07.587281+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:32.722933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.569120+0000","last_change":"2024-09-12T01:34:07.565143+0000","last_active":"2024-09-12T01:34:08.569120+0000","last_peered":"2024-09-12T01:34:08.569120+0000","last_clean":"2024-09-12T01:34:08.569120+0000","last_became_active":"2024-09-12T01:34:07.564860+0000","last_became_peered":"2024-09-12T01:34:07.564860+0000","last_unstale":"2024-09-12T01:34:08.569120+0000","last_undegraded":"2024-09-12T01:34:08.569120+0000","last_fullsized":"2024-09-12T01:34:08.569120+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:42.531651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.705722+0000","last_change":"2024-09-12T01:34:07.577322+0000","last_active":"2024-09-12T01:34:09.705722+0000","last_peered":"2024-09-12T01:34:09.705722+0000","last_clean":"2024-09-12T01:34:09.705722+0000","last_became_active":"2024-09-12T01:34:07.577063+0000","last_became_peered":"2024-09-12T01:34:07.577063+0000","last_unstale":"2024-09-12T01:34:09.705722+0000","last_undegraded":"2024-09-12T01:34:09.705722+0000","last_fullsized":"2024-09-12T01:34:09.705722+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:53.456847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.659181+0000","last_change":"2024-09-12T01:34:07.570352+0000","last_active":"2024-09-12T01:34:08.659181+0000","last_peered":"2024-09-12T01:34:08.659181+0000","last_clean":"2024-09-12T01:34:08.659181+0000","last_became_active":"2024-09-12T01:34:07.569630+0000","last_became_peered":"2024-09-12T01:34:07.569630+0000","last_unstale":"2024-09-12T01:34:08.659181+0000","last_undegraded":"2024-09-12T01:34:08.659181+0000","last_fullsized":"2024-09-12T01:34:08.659181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:36:53.435783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:10.093181+0000","last_change":"2024-09-12T01:34:07.575108+0000","last_active":"2024-09-12T01:34:10.093181+0000","last_peered":"2024-09-12T01:34:10.093181+0000","last_clean":"2024-09-12T01:34:10.093181+0000","last_became_active":"2024-09-12T01:34:07.574824+0000","last_became_peered":"2024-09-12T01:34:07.574824+0000","last_unstale":"2024-09-12T01:34:10.093181+0000","last_undegraded":"2024-09-12T01:34:10.093181+0000","last_fullsized":"2024-09-12T01:34:10.093181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:28.448686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584594+0000","last_change":"2024-09-12T01:34:07.583580+0000","last_active":"2024-09-12T01:34:07.584594+0000","last_peered":"2024-09-12T01:34:07.584594+0000","last_clean":"2024-09-12T01:34:07.584594+0000","last_became_active":"2024-09-12T01:34:07.580853+0000","last_became_peered":"2024-09-12T01:34:07.580853+0000","last_unstale":"2024-09-12T01:34:07.584594+0000","last_undegraded":"2024-09-12T01:34:07.584594+0000","last_fullsized":"2024-09-12T01:34:07.584594+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:21.696603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:10.093058+0000","last_change":"2024-09-12T01:34:07.569567+0000","last_active":"2024-09-12T01:34:10.093058+0000","last_peered":"2024-09-12T01:34:10.093058+0000","last_clean":"2024-09-12T01:34:10.093058+0000","last_became_active":"2024-09-12T01:34:07.569272+0000","last_became_peered":"2024-09-12T01:34:07.569272+0000","last_unstale":"2024-09-12T01:34:10.093058+0000","last_undegraded":"2024-09-12T01:34:10.093058+0000","last_fullsized":"2024-09-12T01:34:10.093058+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:32.726522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.593798+0000","last_change":"2024-09-12T01:34:07.588993+0000","last_active":"2024-09-12T01:34:07.593798+0000","last_peered":"2024-09-12T01:34:07.593798+0000","last_clean":"2024-09-12T01:34:07.593798+0000","last_became_active":"2024-09-12T01:34:07.587401+0000","last_became_peered":"2024-09-12T01:34:07.587401+0000","last_unstale":"2024-09-12T01:34:07.593798+0000","last_undegraded":"2024-09-12T01:34:07.593798+0000","last_fullsized":"2024-09-12T01:34:07.593798+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:42.531651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:10.092918+0000","last_change":"2024-09-12T01:34:07.570329+0000","last_active":"2024-09-12T01:34:10.092918+0000","last_peered":"2024-09-12T01:34:10.092918+0000","last_clean":"2024-09-12T01:34:10.092918+0000","last_became_active":"2024-09-12T01:34:07.570048+0000","last_became_peered":"2024-09-12T01:34:07.570048+0000","last_unstale":"2024-09-12T01:34:10.092918+0000","last_undegraded":"2024-09-12T01:34:10.092918+0000","last_fullsized":"2024-09-12T01:34:10.092918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:03.055098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:10.092813+0000","last_change":"2024-09-12T01:34:07.575895+0000","last_active":"2024-09-12T01:34:10.092813+0000","last_peered":"2024-09-12T01:34:10.092813+0000","last_clean":"2024-09-12T01:34:10.092813+0000","last_became_active":"2024-09-12T01:34:07.575620+0000","last_became_peered":"2024-09-12T01:34:07.575620+0000","last_unstale":"2024-09-12T01:34:10.092813+0000","last_undegraded":"2024-09-12T01:34:10.092813+0000","last_fullsized":"2024-09-12T01:34:10.092813+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:19.574669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.705803+0000","last_change":"2024-09-12T01:34:07.578040+0000","last_active":"2024-09-12T01:34:09.705803+0000","last_peered":"2024-09-12T01:34:09.705803+0000","last_clean":"2024-09-12T01:34:09.705803+0000","last_became_active":"2024-09-12T01:34:07.577773+0000","last_became_peered":"2024-09-12T01:34:07.577773+0000","last_unstale":"2024-09-12T01:34:09.705803+0000","last_undegraded":"2024-09-12T01:34:09.705803+0000","last_fullsized":"2024-09-12T01:34:09.705803+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:42.531651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584661+0000","last_change":"2024-09-12T01:34:07.580355+0000","last_active":"2024-09-12T01:34:07.584661+0000","last_peered":"2024-09-12T01:34:07.584661+0000","last_clean":"2024-09-12T01:34:07.584661+0000","last_became_active":"2024-09-12T01:34:07.580020+0000","last_became_peered":"2024-09-12T01:34:07.580020+0000","last_unstale":"2024-09-12T01:34:07.584661+0000","last_undegraded":"2024-09-12T01:34:07.584661+0000","last_fullsized":"2024-09-12T01:34:07.584661+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:22.777538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.705962+0000","last_change":"2024-09-12T01:34:07.583460+0000","last_active":"2024-09-12T01:34:09.705962+0000","last_peered":"2024-09-12T01:34:09.705962+0000","last_clean":"2024-09-12T01:34:09.705962+0000","last_became_active":"2024-09-12T01:34:07.583158+0000","last_became_peered":"2024-09-12T01:34:07.583158+0000","last_unstale":"2024-09-12T01:34:09.705962+0000","last_undegraded":"2024-09-12T01:34:09.705962+0000","last_fullsized":"2024-09-12T01:34:09.705962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:53.631510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:10.093234+0000","last_change":"2024-09-12T01:34:06.573844+0000","last_active":"2024-09-12T01:34:10.093234+0000","last_peered":"2024-09-12T01:34:10.093234+0000","last_clean":"2024-09-12T01:34:10.093234+0000","last_became_active":"2024-09-12T01:34:06.573556+0000","last_became_peered":"2024-09-12T01:34:06.573556+0000","last_unstale":"2024-09-12T01:34:10.093234+0000","last_undegraded":"2024-09-12T01:34:10.093234+0000","last_fullsized":"2024-09-12T01:34:10.093234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:00.880312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584013+0000","last_change":"2024-09-12T01:34:07.581833+0000","last_active":"2024-09-12T01:34:07.584013+0000","last_peered":"2024-09-12T01:34:07.584013+0000","last_clean":"2024-09-12T01:34:07.584013+0000","last_became_active":"2024-09-12T01:34:07.581526+0000","last_became_peered":"2024-09-12T01:34:07.581526+0000","last_unstale":"2024-09-12T01:34:07.584013+0000","last_undegraded":"2024-09-12T01:34:07.584013+0000","last_fullsized":"2024-09-12T01:34:07.584013+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:28.448686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.009860+0000","last_change":"2024-09-12T01:34:06.572433+0000","last_active":"2024-09-12T01:34:09.009860+0000","last_peered":"2024-09-12T01:34:09.009860+0000","last_clean":"2024-09-12T01:34:09.009860+0000","last_became_active":"2024-09-12T01:34:06.572147+0000","last_became_peered":"2024-09-12T01:34:06.572147+0000","last_unstale":"2024-09-12T01:34:09.009860+0000","last_undegraded":"2024-09-12T01:34:09.009860+0000","last_fullsized":"2024-09-12T01:34:09.009860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:33.031885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.569283+0000","last_change":"2024-09-12T01:34:07.577018+0000","last_active":"2024-09-12T01:34:08.569283+0000","last_peered":"2024-09-12T01:34:08.569283+0000","last_clean":"2024-09-12T01:34:08.569283+0000","last_became_active":"2024-09-12T01:34:07.576118+0000","last_became_peered":"2024-09-12T01:34:07.576118+0000","last_unstale":"2024-09-12T01:34:08.569283+0000","last_undegraded":"2024-09-12T01:34:08.569283+0000","last_fullsized":"2024-09-12T01:34:08.569283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:53.631510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.581274+0000","last_change":"2024-09-12T01:34:06.573185+0000","last_active":"2024-09-12T01:34:07.581274+0000","last_peered":"2024-09-12T01:34:07.581274+0000","last_clean":"2024-09-12T01:34:07.581274+0000","last_became_active":"2024-09-12T01:34:06.572814+0000","last_became_peered":"2024-09-12T01:34:06.572814+0000","last_unstale":"2024-09-12T01:34:07.581274+0000","last_undegraded":"2024-09-12T01:34:07.581274+0000","last_fullsized":"2024-09-12T01:34:07.581274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:35.191411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568251+0000","last_change":"2024-09-12T01:34:07.575202+0000","last_active":"2024-09-12T01:34:08.568251+0000","last_peered":"2024-09-12T01:34:08.568251+0000","last_clean":"2024-09-12T01:34:08.568251+0000","last_became_active":"2024-09-12T01:34:07.574888+0000","last_became_peered":"2024-09-12T01:34:07.574888+0000","last_unstale":"2024-09-12T01:34:08.568251+0000","last_undegraded":"2024-09-12T01:34:08.568251+0000","last_fullsized":"2024-09-12T01:34:08.568251+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:17.171969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.569207+0000","last_change":"2024-09-12T01:34:06.571688+0000","last_active":"2024-09-12T01:34:08.569207+0000","last_peered":"2024-09-12T01:34:08.569207+0000","last_clean":"2024-09-12T01:34:08.569207+0000","last_became_active":"2024-09-12T01:34:06.571103+0000","last_became_peered":"2024-09-12T01:34:06.571103+0000","last_unstale":"2024-09-12T01:34:08.569207+0000","last_undegraded":"2024-09-12T01:34:08.569207+0000","last_fullsized":"2024-09-12T01:34:08.569207+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:18.569840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.009616+0000","last_change":"2024-09-12T01:34:07.582834+0000","last_active":"2024-09-12T01:34:09.009616+0000","last_peered":"2024-09-12T01:34:09.009616+0000","last_clean":"2024-09-12T01:34:09.009616+0000","last_became_active":"2024-09-12T01:34:07.582542+0000","last_became_peered":"2024-09-12T01:34:07.582542+0000","last_unstale":"2024-09-12T01:34:09.009616+0000","last_undegraded":"2024-09-12T01:34:09.009616+0000","last_fullsized":"2024-09-12T01:34:09.009616+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:57.100216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.590117+0000","last_change":"2024-09-12T01:34:06.573600+0000","last_active":"2024-09-12T01:34:07.590117+0000","last_peered":"2024-09-12T01:34:07.590117+0000","last_clean":"2024-09-12T01:34:07.590117+0000","last_became_active":"2024-09-12T01:34:06.572841+0000","last_became_peered":"2024-09-12T01:34:06.572841+0000","last_unstale":"2024-09-12T01:34:07.590117+0000","last_undegraded":"2024-09-12T01:34:07.590117+0000","last_fullsized":"2024-09-12T01:34:07.590117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:38.064258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.584940+0000","last_change":"2024-09-12T01:34:07.581469+0000","last_active":"2024-09-12T01:34:07.584940+0000","last_peered":"2024-09-12T01:34:07.584940+0000","last_clean":"2024-09-12T01:34:07.584940+0000","last_became_active":"2024-09-12T01:34:07.581163+0000","last_became_peered":"2024-09-12T01:34:07.581163+0000","last_unstale":"2024-09-12T01:34:07.584940+0000","last_undegraded":"2024-09-12T01:34:07.584940+0000","last_fullsized":"2024-09-12T01:34:07.584940+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:59:31.150513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.705670+0000","last_change":"2024-09-12T01:34:06.573927+0000","last_active":"2024-09-12T01:34:09.705670+0000","last_peered":"2024-09-12T01:34:09.705670+0000","last_clean":"2024-09-12T01:34:09.705670+0000","last_became_active":"2024-09-12T01:34:06.573608+0000","last_became_peered":"2024-09-12T01:34:06.573608+0000","last_unstale":"2024-09-12T01:34:09.705670+0000","last_undegraded":"2024-09-12T01:34:09.705670+0000","last_fullsized":"2024-09-12T01:34:09.705670+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:18.183678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.569169+0000","last_change":"2024-09-12T01:34:07.582019+0000","last_active":"2024-09-12T01:34:08.569169+0000","last_peered":"2024-09-12T01:34:08.569169+0000","last_clean":"2024-09-12T01:34:08.569169+0000","last_became_active":"2024-09-12T01:34:07.581722+0000","last_became_peered":"2024-09-12T01:34:07.581722+0000","last_unstale":"2024-09-12T01:34:08.569169+0000","last_undegraded":"2024-09-12T01:34:08.569169+0000","last_fullsized":"2024-09-12T01:34:08.569169+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:29.720685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562765+0000","last_change":"2024-09-12T01:34:06.571529+0000","last_active":"2024-09-12T01:34:07.562765+0000","last_peered":"2024-09-12T01:34:07.562765+0000","last_clean":"2024-09-12T01:34:07.562765+0000","last_became_active":"2024-09-12T01:34:06.571192+0000","last_became_peered":"2024-09-12T01:34:06.571192+0000","last_unstale":"2024-09-12T01:34:07.562765+0000","last_undegraded":"2024-09-12T01:34:07.562765+0000","last_fullsized":"2024-09-12T01:34:07.562765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:18.183678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568020+0000","last_change":"2024-09-12T01:34:07.572915+0000","last_active":"2024-09-12T01:34:08.568020+0000","last_peered":"2024-09-12T01:34:08.568020+0000","last_clean":"2024-09-12T01:34:08.568020+0000","last_became_active":"2024-09-12T01:34:07.572585+0000","last_became_peered":"2024-09-12T01:34:07.572585+0000","last_unstale":"2024-09-12T01:34:08.568020+0000","last_undegraded":"2024-09-12T01:34:08.568020+0000","last_fullsized":"2024-09-12T01:34:08.568020+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:31.058724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568575+0000","last_change":"2024-09-12T01:34:06.575351+0000","last_active":"2024-09-12T01:34:08.568575+0000","last_peered":"2024-09-12T01:34:08.568575+0000","last_clean":"2024-09-12T01:34:08.568575+0000","last_became_active":"2024-09-12T01:34:06.575047+0000","last_became_peered":"2024-09-12T01:34:06.575047+0000","last_unstale":"2024-09-12T01:34:08.568575+0000","last_undegraded":"2024-09-12T01:34:08.568575+0000","last_fullsized":"2024-09-12T01:34:08.568575+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:35.191411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.659443+0000","last_change":"2024-09-12T01:34:07.573978+0000","last_active":"2024-09-12T01:34:08.659443+0000","last_peered":"2024-09-12T01:34:08.659443+0000","last_clean":"2024-09-12T01:34:08.659443+0000","last_became_active":"2024-09-12T01:34:07.573412+0000","last_became_peered":"2024-09-12T01:34:07.573412+0000","last_unstale":"2024-09-12T01:34:08.659443+0000","last_undegraded":"2024-09-12T01:34:08.659443+0000","last_fullsized":"2024-09-12T01:34:08.659443+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:11.870971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568445+0000","last_change":"2024-09-12T01:34:06.566440+0000","last_active":"2024-09-12T01:34:08.568445+0000","last_peered":"2024-09-12T01:34:08.568445+0000","last_clean":"2024-09-12T01:34:08.568445+0000","last_became_active":"2024-09-12T01:34:06.566046+0000","last_became_peered":"2024-09-12T01:34:06.566046+0000","last_unstale":"2024-09-12T01:34:08.568445+0000","last_undegraded":"2024-09-12T01:34:08.568445+0000","last_fullsized":"2024-09-12T01:34:08.568445+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:27.443857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.706060+0000","last_change":"2024-09-12T01:34:07.572877+0000","last_active":"2024-09-12T01:34:09.706060+0000","last_peered":"2024-09-12T01:34:09.706060+0000","last_clean":"2024-09-12T01:34:09.706060+0000","last_became_active":"2024-09-12T01:34:07.572547+0000","last_became_peered":"2024-09-12T01:34:07.572547+0000","last_unstale":"2024-09-12T01:34:09.706060+0000","last_undegraded":"2024-09-12T01:34:09.706060+0000","last_fullsized":"2024-09-12T01:34:09.706060+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:51:11.203412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.570282+0000","last_change":"2024-09-12T01:34:06.577223+0000","last_active":"2024-09-12T01:34:07.570282+0000","last_peered":"2024-09-12T01:34:07.570282+0000","last_clean":"2024-09-12T01:34:07.570282+0000","last_became_active":"2024-09-12T01:34:06.576653+0000","last_became_peered":"2024-09-12T01:34:06.576653+0000","last_unstale":"2024-09-12T01:34:07.570282+0000","last_undegraded":"2024-09-12T01:34:07.570282+0000","last_fullsized":"2024-09-12T01:34:07.570282+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:35.191411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.009874+0000","last_change":"2024-09-12T01:34:07.575180+0000","last_active":"2024-09-12T01:34:09.009874+0000","last_peered":"2024-09-12T01:34:09.009874+0000","last_clean":"2024-09-12T01:34:09.009874+0000","last_became_active":"2024-09-12T01:34:07.574648+0000","last_became_peered":"2024-09-12T01:34:07.574648+0000","last_unstale":"2024-09-12T01:34:09.009874+0000","last_undegraded":"2024-09-12T01:34:09.009874+0000","last_fullsized":"2024-09-12T01:34:09.009874+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:01.083559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.590306+0000","last_change":"2024-09-12T01:34:06.576881+0000","last_active":"2024-09-12T01:34:07.590306+0000","last_peered":"2024-09-12T01:34:07.590306+0000","last_clean":"2024-09-12T01:34:07.590306+0000","last_became_active":"2024-09-12T01:34:06.576580+0000","last_became_peered":"2024-09-12T01:34:06.576580+0000","last_unstale":"2024-09-12T01:34:07.590306+0000","last_undegraded":"2024-09-12T01:34:07.590306+0000","last_fullsized":"2024-09-12T01:34:07.590306+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:18.569840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.569279+0000","last_change":"2024-09-12T01:34:07.574317+0000","last_active":"2024-09-12T01:34:08.569279+0000","last_peered":"2024-09-12T01:34:08.569279+0000","last_clean":"2024-09-12T01:34:08.569279+0000","last_became_active":"2024-09-12T01:34:07.573968+0000","last_became_peered":"2024-09-12T01:34:07.573968+0000","last_unstale":"2024-09-12T01:34:08.569279+0000","last_undegraded":"2024-09-12T01:34:08.569279+0000","last_fullsized":"2024-09-12T01:34:08.569279+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:32.486554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568357+0000","last_change":"2024-09-12T01:34:06.576115+0000","last_active":"2024-09-12T01:34:08.568357+0000","last_peered":"2024-09-12T01:34:08.568357+0000","last_clean":"2024-09-12T01:34:08.568357+0000","last_became_active":"2024-09-12T01:34:06.575873+0000","last_became_peered":"2024-09-12T01:34:06.575873+0000","last_unstale":"2024-09-12T01:34:08.568357+0000","last_undegraded":"2024-09-12T01:34:08.568357+0000","last_fullsized":"2024-09-12T01:34:08.568357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:20.691775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.009670+0000","last_change":"2024-09-12T01:34:07.588111+0000","last_active":"2024-09-12T01:34:09.009670+0000","last_peered":"2024-09-12T01:34:09.009670+0000","last_clean":"2024-09-12T01:34:09.009670+0000","last_became_active":"2024-09-12T01:34:07.587822+0000","last_became_peered":"2024-09-12T01:34:07.587822+0000","last_unstale":"2024-09-12T01:34:09.009670+0000","last_undegraded":"2024-09-12T01:34:09.009670+0000","last_fullsized":"2024-09-12T01:34:09.009670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:24:43.100606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568240+0000","last_change":"2024-09-12T01:34:06.566516+0000","last_active":"2024-09-12T01:34:08.568240+0000","last_peered":"2024-09-12T01:34:08.568240+0000","last_clean":"2024-09-12T01:34:08.568240+0000","last_became_active":"2024-09-12T01:34:06.566186+0000","last_became_peered":"2024-09-12T01:34:06.566186+0000","last_unstale":"2024-09-12T01:34:08.568240+0000","last_undegraded":"2024-09-12T01:34:08.568240+0000","last_fullsized":"2024-09-12T01:34:08.568240+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:59:30.145684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567848+0000","last_change":"2024-09-12T01:34:07.567627+0000","last_active":"2024-09-12T01:34:08.567848+0000","last_peered":"2024-09-12T01:34:08.567848+0000","last_clean":"2024-09-12T01:34:08.567848+0000","last_became_active":"2024-09-12T01:34:07.567381+0000","last_became_peered":"2024-09-12T01:34:07.567381+0000","last_unstale":"2024-09-12T01:34:08.567848+0000","last_undegraded":"2024-09-12T01:34:08.567848+0000","last_fullsized":"2024-09-12T01:34:08.567848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:30.783361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562642+0000","last_change":"2024-09-12T01:34:06.573250+0000","last_active":"2024-09-12T01:34:07.562642+0000","last_peered":"2024-09-12T01:34:07.562642+0000","last_clean":"2024-09-12T01:34:07.562642+0000","last_became_active":"2024-09-12T01:34:06.572929+0000","last_became_peered":"2024-09-12T01:34:06.572929+0000","last_unstale":"2024-09-12T01:34:07.562642+0000","last_undegraded":"2024-09-12T01:34:07.562642+0000","last_fullsized":"2024-09-12T01:34:07.562642+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:43.363051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.705958+0000","last_change":"2024-09-12T01:34:07.576600+0000","last_active":"2024-09-12T01:34:09.705958+0000","last_peered":"2024-09-12T01:34:09.705958+0000","last_clean":"2024-09-12T01:34:09.705958+0000","last_became_active":"2024-09-12T01:34:07.576321+0000","last_became_peered":"2024-09-12T01:34:07.576321+0000","last_unstale":"2024-09-12T01:34:09.705958+0000","last_undegraded":"2024-09-12T01:34:09.705958+0000","last_fullsized":"2024-09-12T01:34:09.705958+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:32.486554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:10.092699+0000","last_change":"2024-09-12T01:34:06.573139+0000","last_active":"2024-09-12T01:34:10.092699+0000","last_peered":"2024-09-12T01:34:10.092699+0000","last_clean":"2024-09-12T01:34:10.092699+0000","last_became_active":"2024-09-12T01:34:06.572850+0000","last_became_peered":"2024-09-12T01:34:06.572850+0000","last_unstale":"2024-09-12T01:34:10.092699+0000","last_undegraded":"2024-09-12T01:34:10.092699+0000","last_fullsized":"2024-09-12T01:34:10.092699+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:05.816134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568123+0000","last_change":"2024-09-12T01:34:07.566974+0000","last_active":"2024-09-12T01:34:08.568123+0000","last_peered":"2024-09-12T01:34:08.568123+0000","last_clean":"2024-09-12T01:34:08.568123+0000","last_became_active":"2024-09-12T01:34:07.566688+0000","last_became_peered":"2024-09-12T01:34:07.566688+0000","last_unstale":"2024-09-12T01:34:08.568123+0000","last_undegraded":"2024-09-12T01:34:08.568123+0000","last_fullsized":"2024-09-12T01:34:08.568123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:35.009027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.705623+0000","last_change":"2024-09-12T01:34:06.572958+0000","last_active":"2024-09-12T01:34:09.705623+0000","last_peered":"2024-09-12T01:34:09.705623+0000","last_clean":"2024-09-12T01:34:09.705623+0000","last_became_active":"2024-09-12T01:34:06.572616+0000","last_became_peered":"2024-09-12T01:34:06.572616+0000","last_unstale":"2024-09-12T01:34:09.705623+0000","last_undegraded":"2024-09-12T01:34:09.705623+0000","last_fullsized":"2024-09-12T01:34:09.705623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:43.363051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568341+0000","last_change":"2024-09-12T01:34:07.576753+0000","last_active":"2024-09-12T01:34:08.568341+0000","last_peered":"2024-09-12T01:34:08.568341+0000","last_clean":"2024-09-12T01:34:08.568341+0000","last_became_active":"2024-09-12T01:34:07.576461+0000","last_became_peered":"2024-09-12T01:34:07.576461+0000","last_unstale":"2024-09-12T01:34:08.568341+0000","last_undegraded":"2024-09-12T01:34:08.568341+0000","last_fullsized":"2024-09-12T01:34:08.568341+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:12.831326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.591564+0000","last_change":"2024-09-12T01:34:06.580294+0000","last_active":"2024-09-12T01:34:07.591564+0000","last_peered":"2024-09-12T01:34:07.591564+0000","last_clean":"2024-09-12T01:34:07.591564+0000","last_became_active":"2024-09-12T01:34:06.580015+0000","last_became_peered":"2024-09-12T01:34:06.580015+0000","last_unstale":"2024-09-12T01:34:07.591564+0000","last_undegraded":"2024-09-12T01:34:07.591564+0000","last_fullsized":"2024-09-12T01:34:07.591564+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:21.772709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.587692+0000","last_change":"2024-09-12T01:34:07.584456+0000","last_active":"2024-09-12T01:34:07.587692+0000","last_peered":"2024-09-12T01:34:07.587692+0000","last_clean":"2024-09-12T01:34:07.587692+0000","last_became_active":"2024-09-12T01:34:07.584160+0000","last_became_peered":"2024-09-12T01:34:07.584160+0000","last_unstale":"2024-09-12T01:34:07.587692+0000","last_undegraded":"2024-09-12T01:34:07.587692+0000","last_fullsized":"2024-09-12T01:34:07.587692+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:15.555273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.569382+0000","last_change":"2024-09-12T01:34:06.579860+0000","last_active":"2024-09-12T01:34:08.569382+0000","last_peered":"2024-09-12T01:34:08.569382+0000","last_clean":"2024-09-12T01:34:08.569382+0000","last_became_active":"2024-09-12T01:34:06.579515+0000","last_became_peered":"2024-09-12T01:34:06.579515+0000","last_unstale":"2024-09-12T01:34:08.569382+0000","last_undegraded":"2024-09-12T01:34:08.569382+0000","last_fullsized":"2024-09-12T01:34:08.569382+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:02.050269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.569244+0000","last_change":"2024-09-12T01:34:07.572646+0000","last_active":"2024-09-12T01:34:08.569244+0000","last_peered":"2024-09-12T01:34:08.569244+0000","last_clean":"2024-09-12T01:34:08.569244+0000","last_became_active":"2024-09-12T01:34:07.572383+0000","last_became_peered":"2024-09-12T01:34:07.572383+0000","last_unstale":"2024-09-12T01:34:08.569244+0000","last_undegraded":"2024-09-12T01:34:08.569244+0000","last_fullsized":"2024-09-12T01:34:08.569244+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:25.387081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.659479+0000","last_change":"2024-09-12T01:34:06.572508+0000","last_active":"2024-09-12T01:34:08.659479+0000","last_peered":"2024-09-12T01:34:08.659479+0000","last_clean":"2024-09-12T01:34:08.659479+0000","last_became_active":"2024-09-12T01:34:06.572222+0000","last_became_peered":"2024-09-12T01:34:06.572222+0000","last_unstale":"2024-09-12T01:34:08.659479+0000","last_undegraded":"2024-09-12T01:34:08.659479+0000","last_fullsized":"2024-09-12T01:34:08.659479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:12.959609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.659567+0000","last_change":"2024-09-12T01:34:07.571161+0000","last_active":"2024-09-12T01:34:08.659567+0000","last_peered":"2024-09-12T01:34:08.659567+0000","last_clean":"2024-09-12T01:34:08.659567+0000","last_became_active":"2024-09-12T01:34:07.570891+0000","last_became_peered":"2024-09-12T01:34:07.570891+0000","last_unstale":"2024-09-12T01:34:08.659567+0000","last_undegraded":"2024-09-12T01:34:08.659567+0000","last_fullsized":"2024-09-12T01:34:08.659567+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:53.113392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.562901+0000","last_change":"2024-09-12T01:34:06.574086+0000","last_active":"2024-09-12T01:34:07.562901+0000","last_peered":"2024-09-12T01:34:07.562901+0000","last_clean":"2024-09-12T01:34:07.562901+0000","last_became_active":"2024-09-12T01:34:06.573682+0000","last_became_peered":"2024-09-12T01:34:06.573682+0000","last_unstale":"2024-09-12T01:34:07.562901+0000","last_undegraded":"2024-09-12T01:34:07.562901+0000","last_fullsized":"2024-09-12T01:34:07.562901+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:38.064258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.705859+0000","last_change":"2024-09-12T01:34:07.575968+0000","last_active":"2024-09-12T01:34:09.705859+0000","last_peered":"2024-09-12T01:34:09.705859+0000","last_clean":"2024-09-12T01:34:09.705859+0000","last_became_active":"2024-09-12T01:34:07.575689+0000","last_became_peered":"2024-09-12T01:34:07.575689+0000","last_unstale":"2024-09-12T01:34:09.705859+0000","last_undegraded":"2024-09-12T01:34:09.705859+0000","last_fullsized":"2024-09-12T01:34:09.705859+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:29.720685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.659223+0000","last_change":"2024-09-12T01:34:06.563081+0000","last_active":"2024-09-12T01:34:08.659223+0000","last_peered":"2024-09-12T01:34:08.659223+0000","last_clean":"2024-09-12T01:34:08.659223+0000","last_became_active":"2024-09-12T01:34:06.562728+0000","last_became_peered":"2024-09-12T01:34:06.562728+0000","last_unstale":"2024-09-12T01:34:08.659223+0000","last_undegraded":"2024-09-12T01:34:08.659223+0000","last_fullsized":"2024-09-12T01:34:08.659223+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:56.551827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.009692+0000","last_change":"2024-09-12T01:34:07.575362+0000","last_active":"2024-09-12T01:34:09.009692+0000","last_peered":"2024-09-12T01:34:09.009692+0000","last_clean":"2024-09-12T01:34:09.009692+0000","last_became_active":"2024-09-12T01:34:07.574340+0000","last_became_peered":"2024-09-12T01:34:07.574340+0000","last_unstale":"2024-09-12T01:34:09.009692+0000","last_undegraded":"2024-09-12T01:34:09.009692+0000","last_fullsized":"2024-09-12T01:34:09.009692+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:11.066234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.659250+0000","last_change":"2024-09-12T01:34:06.573266+0000","last_active":"2024-09-12T01:34:08.659250+0000","last_peered":"2024-09-12T01:34:08.659250+0000","last_clean":"2024-09-12T01:34:08.659250+0000","last_became_active":"2024-09-12T01:34:06.572679+0000","last_became_peered":"2024-09-12T01:34:06.572679+0000","last_unstale":"2024-09-12T01:34:08.659250+0000","last_undegraded":"2024-09-12T01:34:08.659250+0000","last_fullsized":"2024-09-12T01:34:08.659250+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:50.930775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.585466+0000","last_change":"2024-09-12T01:34:07.580371+0000","last_active":"2024-09-12T01:34:07.585466+0000","last_peered":"2024-09-12T01:34:07.585466+0000","last_clean":"2024-09-12T01:34:07.585466+0000","last_became_active":"2024-09-12T01:34:07.579990+0000","last_became_peered":"2024-09-12T01:34:07.579990+0000","last_unstale":"2024-09-12T01:34:07.585466+0000","last_undegraded":"2024-09-12T01:34:07.585466+0000","last_fullsized":"2024-09-12T01:34:07.585466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:32.726522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.705938+0000","last_change":"2024-09-12T01:34:06.574756+0000","last_active":"2024-09-12T01:34:09.705938+0000","last_peered":"2024-09-12T01:34:09.705938+0000","last_clean":"2024-09-12T01:34:09.705938+0000","last_became_active":"2024-09-12T01:34:06.574370+0000","last_became_peered":"2024-09-12T01:34:06.574370+0000","last_unstale":"2024-09-12T01:34:09.705938+0000","last_undegraded":"2024-09-12T01:34:09.705938+0000","last_fullsized":"2024-09-12T01:34:09.705938+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:51.653996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568150+0000","last_change":"2024-09-12T01:34:07.572416+0000","last_active":"2024-09-12T01:34:08.568150+0000","last_peered":"2024-09-12T01:34:08.568150+0000","last_clean":"2024-09-12T01:34:08.568150+0000","last_became_active":"2024-09-12T01:34:07.572092+0000","last_became_peered":"2024-09-12T01:34:07.572092+0000","last_unstale":"2024-09-12T01:34:08.568150+0000","last_undegraded":"2024-09-12T01:34:08.568150+0000","last_fullsized":"2024-09-12T01:34:08.568150+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:55.078476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.659137+0000","last_change":"2024-09-12T01:34:06.562339+0000","last_active":"2024-09-12T01:34:08.659137+0000","last_peered":"2024-09-12T01:34:08.659137+0000","last_clean":"2024-09-12T01:34:08.659137+0000","last_became_active":"2024-09-12T01:34:06.561990+0000","last_became_peered":"2024-09-12T01:34:06.561990+0000","last_unstale":"2024-09-12T01:34:08.659137+0000","last_undegraded":"2024-09-12T01:34:08.659137+0000","last_fullsized":"2024-09-12T01:34:08.659137+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:09.175440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.583991+0000","last_change":"2024-09-12T01:34:07.581576+0000","last_active":"2024-09-12T01:34:07.583991+0000","last_peered":"2024-09-12T01:34:07.583991+0000","last_clean":"2024-09-12T01:34:07.583991+0000","last_became_active":"2024-09-12T01:34:07.580839+0000","last_became_peered":"2024-09-12T01:34:07.580839+0000","last_unstale":"2024-09-12T01:34:07.583991+0000","last_undegraded":"2024-09-12T01:34:07.583991+0000","last_fullsized":"2024-09-12T01:34:07.583991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:32.726522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.590455+0000","last_change":"2024-09-12T01:34:06.577402+0000","last_active":"2024-09-12T01:34:07.590455+0000","last_peered":"2024-09-12T01:34:07.590455+0000","last_clean":"2024-09-12T01:34:07.590455+0000","last_became_active":"2024-09-12T01:34:06.577055+0000","last_became_peered":"2024-09-12T01:34:06.577055+0000","last_unstale":"2024-09-12T01:34:07.590455+0000","last_undegraded":"2024-09-12T01:34:07.590455+0000","last_fullsized":"2024-09-12T01:34:07.590455+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:27.443857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.706142+0000","last_change":"2024-09-12T01:34:07.573076+0000","last_active":"2024-09-12T01:34:09.706142+0000","last_peered":"2024-09-12T01:34:09.706142+0000","last_clean":"2024-09-12T01:34:09.706142+0000","last_became_active":"2024-09-12T01:34:07.572752+0000","last_became_peered":"2024-09-12T01:34:07.572752+0000","last_unstale":"2024-09-12T01:34:09.706142+0000","last_undegraded":"2024-09-12T01:34:09.706142+0000","last_fullsized":"2024-09-12T01:34:09.706142+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:05.998921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.659444+0000","last_change":"2024-09-12T01:34:06.577637+0000","last_active":"2024-09-12T01:34:08.659444+0000","last_peered":"2024-09-12T01:34:08.659444+0000","last_clean":"2024-09-12T01:34:08.659444+0000","last_became_active":"2024-09-12T01:34:06.577338+0000","last_became_peered":"2024-09-12T01:34:06.577338+0000","last_unstale":"2024-09-12T01:34:08.659444+0000","last_undegraded":"2024-09-12T01:34:08.659444+0000","last_fullsized":"2024-09-12T01:34:08.659444+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:44.282477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567958+0000","last_change":"2024-09-12T01:34:07.563962+0000","last_active":"2024-09-12T01:34:08.567958+0000","last_peered":"2024-09-12T01:34:08.567958+0000","last_clean":"2024-09-12T01:34:08.567958+0000","last_became_active":"2024-09-12T01:34:07.563649+0000","last_became_peered":"2024-09-12T01:34:07.563649+0000","last_unstale":"2024-09-12T01:34:08.567958+0000","last_undegraded":"2024-09-12T01:34:08.567958+0000","last_fullsized":"2024-09-12T01:34:08.567958+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:36:19.605889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.009700+0000","last_change":"2024-09-12T01:34:06.564220+0000","last_active":"2024-09-12T01:34:09.009700+0000","last_peered":"2024-09-12T01:34:09.009700+0000","last_clean":"2024-09-12T01:34:09.009700+0000","last_became_active":"2024-09-12T01:34:06.563912+0000","last_became_peered":"2024-09-12T01:34:06.563912+0000","last_unstale":"2024-09-12T01:34:09.009700+0000","last_undegraded":"2024-09-12T01:34:09.009700+0000","last_fullsized":"2024-09-12T01:34:09.009700+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:59.083715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.009851+0000","last_change":"2024-09-12T01:34:07.572710+0000","last_active":"2024-09-12T01:34:09.009851+0000","last_peered":"2024-09-12T01:34:09.009851+0000","last_clean":"2024-09-12T01:34:09.009851+0000","last_became_active":"2024-09-12T01:34:07.572434+0000","last_became_peered":"2024-09-12T01:34:07.572434+0000","last_unstale":"2024-09-12T01:34:09.009851+0000","last_undegraded":"2024-09-12T01:34:09.009851+0000","last_fullsized":"2024-09-12T01:34:09.009851+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:56.210402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.009953+0000","last_change":"2024-09-12T01:34:06.567684+0000","last_active":"2024-09-12T01:34:09.009953+0000","last_peered":"2024-09-12T01:34:09.009953+0000","last_clean":"2024-09-12T01:34:09.009953+0000","last_became_active":"2024-09-12T01:34:06.567184+0000","last_became_peered":"2024-09-12T01:34:06.567184+0000","last_unstale":"2024-09-12T01:34:09.009953+0000","last_undegraded":"2024-09-12T01:34:09.009953+0000","last_fullsized":"2024-09-12T01:34:09.009953+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:00.045391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.580895+0000","last_change":"2024-09-12T01:34:07.573865+0000","last_active":"2024-09-12T01:34:07.580895+0000","last_peered":"2024-09-12T01:34:07.580895+0000","last_clean":"2024-09-12T01:34:07.580895+0000","last_became_active":"2024-09-12T01:34:07.573578+0000","last_became_peered":"2024-09-12T01:34:07.573578+0000","last_unstale":"2024-09-12T01:34:07.580895+0000","last_undegraded":"2024-09-12T01:34:07.580895+0000","last_fullsized":"2024-09-12T01:34:07.580895+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:03.055098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:10.092757+0000","last_change":"2024-09-12T01:34:06.564915+0000","last_active":"2024-09-12T01:34:10.092757+0000","last_peered":"2024-09-12T01:34:10.092757+0000","last_clean":"2024-09-12T01:34:10.092757+0000","last_became_active":"2024-09-12T01:34:06.564588+0000","last_became_peered":"2024-09-12T01:34:06.564588+0000","last_unstale":"2024-09-12T01:34:10.092757+0000","last_undegraded":"2024-09-12T01:34:10.092757+0000","last_fullsized":"2024-09-12T01:34:10.092757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:17.108858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568682+0000","last_change":"2024-09-12T01:34:07.564503+0000","last_active":"2024-09-12T01:34:08.568682+0000","last_peered":"2024-09-12T01:34:08.568682+0000","last_clean":"2024-09-12T01:34:08.568682+0000","last_became_active":"2024-09-12T01:34:07.563941+0000","last_became_peered":"2024-09-12T01:34:07.563941+0000","last_unstale":"2024-09-12T01:34:08.568682+0000","last_undegraded":"2024-09-12T01:34:08.568682+0000","last_fullsized":"2024-09-12T01:34:08.568682+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:53.456847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.568736+0000","last_change":"2024-09-12T01:34:06.571065+0000","last_active":"2024-09-12T01:34:08.568736+0000","last_peered":"2024-09-12T01:34:08.568736+0000","last_clean":"2024-09-12T01:34:08.568736+0000","last_became_active":"2024-09-12T01:34:06.570715+0000","last_became_peered":"2024-09-12T01:34:06.570715+0000","last_unstale":"2024-09-12T01:34:08.568736+0000","last_undegraded":"2024-09-12T01:34:08.568736+0000","last_fullsized":"2024-09-12T01:34:08.568736+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:38.064258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.705981+0000","last_change":"2024-09-12T01:34:07.574447+0000","last_active":"2024-09-12T01:34:09.705981+0000","last_peered":"2024-09-12T01:34:09.705981+0000","last_clean":"2024-09-12T01:34:09.705981+0000","last_became_active":"2024-09-12T01:34:07.574165+0000","last_became_peered":"2024-09-12T01:34:07.574165+0000","last_unstale":"2024-09-12T01:34:09.705981+0000","last_undegraded":"2024-09-12T01:34:09.705981+0000","last_fullsized":"2024-09-12T01:34:09.705981+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:25.387081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:08.567833+0000","last_change":"2024-09-12T01:34:06.566830+0000","last_active":"2024-09-12T01:34:08.567833+0000","last_peered":"2024-09-12T01:34:08.567833+0000","last_clean":"2024-09-12T01:34:08.567833+0000","last_became_active":"2024-09-12T01:34:06.566321+0000","last_became_peered":"2024-09-12T01:34:06.566321+0000","last_unstale":"2024-09-12T01:34:08.567833+0000","last_undegraded":"2024-09-12T01:34:08.567833+0000","last_fullsized":"2024-09-12T01:34:08.567833+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:31.718104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-12T01:34:07.587590+0000","last_change":"2024-09-12T01:34:07.584240+0000","last_active":"2024-09-12T01:34:07.587590+0000","last_peered":"2024-09-12T01:34:07.587590+0000","last_clean":"2024-09-12T01:34:07.587590+0000","last_became_active":"2024-09-12T01:34:07.581299+0000","last_became_peered":"2024-09-12T01:34:07.581299+0000","last_unstale":"2024-09-12T01:34:07.587590+0000","last_undegraded":"2024-09-12T01:34:07.587590+0000","last_fullsized":"2024-09-12T01:34:07.587590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:06.545580+0000","last_clean_scrub_stamp":"2024-09-12T01:34:06.545580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:00.175875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.706020+0000","last_change":"2024-09-12T01:34:06.573208+0000","last_active":"2024-09-12T01:34:09.706020+0000","last_peered":"2024-09-12T01:34:09.706020+0000","last_clean":"2024-09-12T01:34:09.706020+0000","last_became_active":"2024-09-12T01:34:06.572431+0000","last_became_peered":"2024-09-12T01:34:06.572431+0000","last_unstale":"2024-09-12T01:34:09.706020+0000","last_undegraded":"2024-09-12T01:34:09.706020+0000","last_fullsized":"2024-09-12T01:34:09.706020+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:34:05.540751+0000","last_clean_scrub_stamp":"2024-09-12T01:34:05.540751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:41.013338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":204,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-12T01:34:09.705710+0000","last_change":"2024-09-12T01:33:01.096542+0000","last_active":"2024-09-12T01:34:09.705710+0000","last_peered":"2024-09-12T01:34:09.705710+0000","last_clean":"2024-09-12T01:34:09.705710+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:34:09.705710+0000","last_undegraded":"2024-09-12T01:34:09.705710+0000","last_fullsized":"2024-09-12T01:34:09.705710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_clean_scrub_stamp":"2024-09-12T01:32:58.512553+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:48:58.741813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1052672,"data_stored":785232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":193764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":450560,"data_stored":193294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":200478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":471040,"data_stored":201520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":446464,"data_stored":193272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":458752,"data_stored":199944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1052672,"data_stored":790276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:34:15.749 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T01:34:15.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T01:34:15.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.898+0000 7ff838511640 1 Processor -- start 2024-09-12T01:34:15.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.898+0000 7ff838511640 1 -- start start 2024-09-12T01:34:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.899+0000 7ff838511640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff830167310 0x7ff830169700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.899+0000 7ff838511640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8300d01e0 con 0x7ff8300d43b0 2024-09-12T01:34:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.899+0000 7ff838511640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8300d2d60 con 0x7ff8300d2860 2024-09-12T01:34:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.899+0000 7ff838511640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff8300d3040 con 0x7ff830167310 2024-09-12T01:34:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.899+0000 7ff8362ad640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff8300d43b0 0x7ff8300d4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55122/0 (socket says 172.21.15.16:55122) 2024-09-12T01:34:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.899+0000 7ff8362ad640 1 -- 172.21.15.16:0/4045652210 learned_addr learned my addr 172.21.15.16:0/4045652210 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.899+0000 7ff836aae640 1 --2- 172.21.15.16:0/4045652210 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff830167310 0x7ff830169700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.899+0000 7ff8352ab640 1 -- 172.21.15.16:0/4045652210 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1828937296 0 0) 0x7ff8300d01e0 con 0x7ff8300d43b0 2024-09-12T01:34:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.900+0000 7ff8352ab640 1 -- 172.21.15.16:0/4045652210 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff818003520 con 0x7ff8300d43b0 2024-09-12T01:34:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.900+0000 7ff8352ab640 1 -- 172.21.15.16:0/4045652210 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510264024 0 0) 0x7ff8300d2d60 con 0x7ff8300d2860 2024-09-12T01:34:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.900+0000 7ff8352ab640 1 -- 172.21.15.16:0/4045652210 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8300d01e0 con 0x7ff8300d2860 2024-09-12T01:34:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.900+0000 7ff836aae640 1 -- 172.21.15.16:0/4045652210 >> v1:172.21.15.33:6789/0 conn(0x7ff8300d2860 legacy=0x7ff8300d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.900+0000 7ff836aae640 1 -- 172.21.15.16:0/4045652210 >> v1:172.21.15.16:6789/0 conn(0x7ff8300d43b0 legacy=0x7ff8300d4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.900+0000 7ff836aae640 1 -- 172.21.15.16:0/4045652210 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8300d15e0 con 0x7ff830167310 2024-09-12T01:34:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.900+0000 7ff836aae640 1 --2- 172.21.15.16:0/4045652210 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff830167310 0x7ff830169700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff8280097d0 tx=0x7ff82802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8774236195e3423 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.901+0000 7ff8352ab640 1 -- 172.21.15.16:0/4045652210 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828037070 con 0x7ff830167310 2024-09-12T01:34:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.901+0000 7ff8352ab640 1 -- 172.21.15.16:0/4045652210 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff828032440 con 0x7ff830167310 2024-09-12T01:34:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.901+0000 7ff8352ab640 1 -- 172.21.15.16:0/4045652210 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8280327f0 con 0x7ff830167310 2024-09-12T01:34:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.901+0000 7ff838511640 1 -- 172.21.15.16:0/4045652210 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff830167310 msgr2=0x7ff830169700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.901+0000 7ff838511640 1 --2- 172.21.15.16:0/4045652210 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff830167310 0x7ff830169700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff8280097d0 tx=0x7ff82802a180 comp rx=0 tx=0).stop 2024-09-12T01:34:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.901+0000 7ff838511640 1 -- 172.21.15.16:0/4045652210 shutdown_connections 2024-09-12T01:34:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.901+0000 7ff838511640 1 --2- 172.21.15.16:0/4045652210 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff830167310 0x7ff830169700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.901+0000 7ff838511640 1 -- 172.21.15.16:0/4045652210 >> 172.21.15.16:0/4045652210 conn(0x7ff83008bfc0 msgr2=0x7ff83009d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.901+0000 7ff838511640 1 -- 172.21.15.16:0/4045652210 shutdown_connections 2024-09-12T01:34:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.901+0000 7ff838511640 1 -- 172.21.15.16:0/4045652210 wait complete. 2024-09-12T01:34:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.903+0000 7ff838511640 1 Processor -- start 2024-09-12T01:34:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.903+0000 7ff838511640 1 -- start start 2024-09-12T01:34:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.903+0000 7ff838511640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8300d2860 0x7ff830156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.903+0000 7ff838511640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8300d43b0 0x7ff8300b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.903+0000 7ff838511640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830167310 0x7ff830157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.903+0000 7ff838511640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff83006cfd0 con 0x7ff830167310 2024-09-12T01:34:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.903+0000 7ff838511640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8300d1ba0 con 0x7ff8300d43b0 2024-09-12T01:34:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.903+0000 7ff838511640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff83015e7a0 con 0x7ff8300d2860 2024-09-12T01:34:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.903+0000 7ff836aae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830167310 0x7ff830157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.903+0000 7ff836aae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830167310 0x7ff830157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41480/0 (socket says 172.21.15.16:41480) 2024-09-12T01:34:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff8362ad640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8300d2860 0x7ff830156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:15.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff836aae640 1 -- 172.21.15.16:0/1200087663 learned_addr learned my addr 172.21.15.16:0/1200087663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff835aac640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8300d43b0 0x7ff8300b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff836aae640 1 -- 172.21.15.16:0/1200087663 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8300d2860 msgr2=0x7ff830156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff836aae640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8300d2860 0x7ff830156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff836aae640 1 -- 172.21.15.16:0/1200087663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8300d43b0 msgr2=0x7ff8300b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff836aae640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8300d43b0 0x7ff8300b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff836aae640 1 -- 172.21.15.16:0/1200087663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff830157800 con 0x7ff830167310 2024-09-12T01:34:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff836aae640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830167310 0x7ff830157090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff82802a1c0 tx=0x7ff8280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff82effd640 1 -- 172.21.15.16:0/1200087663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828041020 con 0x7ff830167310 2024-09-12T01:34:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff82effd640 1 -- 172.21.15.16:0/1200087663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82803d6e0 con 0x7ff830167310 2024-09-12T01:34:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff82effd640 1 -- 172.21.15.16:0/1200087663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828037040 con 0x7ff830167310 2024-09-12T01:34:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff838511640 1 -- 172.21.15.16:0/1200087663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff830157ae0 con 0x7ff830167310 2024-09-12T01:34:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.904+0000 7ff838511640 1 -- 172.21.15.16:0/1200087663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8301580a0 con 0x7ff830167310 2024-09-12T01:34:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.905+0000 7ff838511640 1 -- 172.21.15.16:0/1200087663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83006e3c0 con 0x7ff830167310 2024-09-12T01:34:15.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.906+0000 7ff82effd640 1 -- 172.21.15.16:0/1200087663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff828044020 con 0x7ff830167310 2024-09-12T01:34:15.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.906+0000 7ff82effd640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff81c06be30 0x7ff81c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:15.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.906+0000 7ff82effd640 1 -- 172.21.15.16:0/1200087663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ff8280ab610 con 0x7ff830167310 2024-09-12T01:34:15.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.906+0000 7ff8362ad640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff81c06be30 0x7ff81c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:15.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.907+0000 7ff8362ad640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff81c06be30 0x7ff81c06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff8200012e0 tx=0x7ff820001ad0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:15.911+0000 7ff82effd640 1 -- 172.21.15.16:0/1200087663 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff82804b030 con 0x7ff830167310 2024-09-12T01:34:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.152+0000 7ff838511640 1 -- 172.21.15.16:0/1200087663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff8300d4790 con 0x7ff830167310 2024-09-12T01:34:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.154+0000 7ff82effd640 1 -- 172.21.15.16:0/1200087663 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7ff8280797f0 con 0x7ff830167310 2024-09-12T01:34:16.155 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-12T01:34:16.155 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCfROJmhMy2IxAAGVo/xGjEHxUMiQfBrw0mrQ== 2024-09-12T01:34:16.155 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-12T01:34:16.155 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-12T01:34:16.155 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T01:34:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-12T01:34:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.157+0000 7ff838511640 1 -- 172.21.15.16:0/1200087663 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff81c06be30 msgr2=0x7ff81c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.157+0000 7ff838511640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff81c06be30 0x7ff81c06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff8200012e0 tx=0x7ff820001ad0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.157+0000 7ff838511640 1 -- 172.21.15.16:0/1200087663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830167310 msgr2=0x7ff830157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.157+0000 7ff838511640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830167310 0x7ff830157090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff82802a1c0 tx=0x7ff8280046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.157+0000 7ff838511640 1 -- 172.21.15.16:0/1200087663 shutdown_connections 2024-09-12T01:34:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.157+0000 7ff838511640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff81c06be30 0x7ff81c06e2f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.157+0000 7ff838511640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830167310 0x7ff830157090 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.157+0000 7ff838511640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8300d43b0 0x7ff8300b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.157+0000 7ff838511640 1 --2- 172.21.15.16:0/1200087663 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8300d2860 0x7ff830156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.157+0000 7ff838511640 1 -- 172.21.15.16:0/1200087663 >> 172.21.15.16:0/1200087663 conn(0x7ff83008bfc0 msgr2=0x7ff8300b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.157+0000 7ff838511640 1 -- 172.21.15.16:0/1200087663 shutdown_connections 2024-09-12T01:34:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.157+0000 7ff838511640 1 -- 172.21.15.16:0/1200087663 wait complete. 2024-09-12T01:34:16.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:34:16.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.270+0000 7f5337c0e640 1 Processor -- start 2024-09-12T01:34:16.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.271+0000 7f5337c0e640 1 -- start start 2024-09-12T01:34:16.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.271+0000 7f5337c0e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53300b88f0 0x7f5330169150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.271+0000 7f5337c0e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53300d3430 con 0x7f53300d0cc0 2024-09-12T01:34:16.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.271+0000 7f5337c0e640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53300d36d0 con 0x7f533015e510 2024-09-12T01:34:16.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.271+0000 7f5337c0e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f53300d39b0 con 0x7f53300b88f0 2024-09-12T01:34:16.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.271+0000 7f53361ab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53300b88f0 0x7f5330169150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.271+0000 7f53351a9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f53300d0cc0 0x7f53300d1120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49252/0 (socket says 172.21.15.16:49252) 2024-09-12T01:34:16.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.271+0000 7f53351a9640 1 -- 172.21.15.16:0/53607736 learned_addr learned my addr 172.21.15.16:0/53607736 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:16.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.272+0000 7f53361ab640 1 -- 172.21.15.16:0/53607736 >> v1:172.21.15.33:6789/0 conn(0x7f533015e510 legacy=0x7f53300ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.272+0000 7f53361ab640 1 -- 172.21.15.16:0/53607736 >> v1:172.21.15.16:6789/0 conn(0x7f53300d0cc0 legacy=0x7f53300d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.272+0000 7f53361ab640 1 -- 172.21.15.16:0/53607736 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53300d3c90 con 0x7f53300b88f0 2024-09-12T01:34:16.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.272+0000 7f53349a8640 1 -- 172.21.15.16:0/53607736 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4170443437 0 0) 0x7f53300d3430 con 0x7f53300d0cc0 2024-09-12T01:34:16.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.273+0000 7f53361ab640 1 --2- 172.21.15.16:0/53607736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53300b88f0 0x7f5330169150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5320009d90 tx=0x7f532002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afbe561d331e212 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:16.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.273+0000 7f53349a8640 1 -- 172.21.15.16:0/53607736 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320007740 con 0x7f53300b88f0 2024-09-12T01:34:16.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.273+0000 7f53349a8640 1 -- 172.21.15.16:0/53607736 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53200078e0 con 0x7f53300b88f0 2024-09-12T01:34:16.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.273+0000 7f53349a8640 1 -- 172.21.15.16:0/53607736 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320007cb0 con 0x7f53300b88f0 2024-09-12T01:34:16.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.273+0000 7f5337c0e640 1 -- 172.21.15.16:0/53607736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53300b88f0 msgr2=0x7f5330169150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.273+0000 7f5337c0e640 1 --2- 172.21.15.16:0/53607736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53300b88f0 0x7f5330169150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5320009d90 tx=0x7f532002a660 comp rx=0 tx=0).stop 2024-09-12T01:34:16.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.273+0000 7f5337c0e640 1 -- 172.21.15.16:0/53607736 shutdown_connections 2024-09-12T01:34:16.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.273+0000 7f5337c0e640 1 --2- 172.21.15.16:0/53607736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53300b88f0 0x7f5330169150 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.273+0000 7f5337c0e640 1 -- 172.21.15.16:0/53607736 >> 172.21.15.16:0/53607736 conn(0x7f533006d1a0 msgr2=0x7f5330073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:16.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.273+0000 7f5337c0e640 1 -- 172.21.15.16:0/53607736 shutdown_connections 2024-09-12T01:34:16.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.273+0000 7f5337c0e640 1 -- 172.21.15.16:0/53607736 wait complete. 2024-09-12T01:34:16.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.274+0000 7f5337c0e640 1 Processor -- start 2024-09-12T01:34:16.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.275+0000 7f5337c0e640 1 -- start start 2024-09-12T01:34:16.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.275+0000 7f5337c0e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53300b88f0 0x7f53300ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.275+0000 7f5337c0e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f53300d0cc0 0x7f53300ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.275+0000 7f5337c0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533015e510 0x7f53300cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.275+0000 7f5337c0e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53300cf210 con 0x7f533015e510 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.275+0000 7f5337c0e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f53300d22a0 con 0x7f53300b88f0 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.275+0000 7f5337c0e640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f533015e8f0 con 0x7f53300d0cc0 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.275+0000 7f53361ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533015e510 0x7f53300cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.275+0000 7f53361ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533015e510 0x7f53300cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45972/0 (socket says 172.21.15.16:45972) 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.275+0000 7f53361ab640 1 -- 172.21.15.16:0/1834809324 learned_addr learned my addr 172.21.15.16:0/1834809324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.276+0000 7f53351a9640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f53300d0cc0 0x7f53300ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.276+0000 7f53361ab640 1 -- 172.21.15.16:0/1834809324 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f53300d0cc0 msgr2=0x7f53300ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.276+0000 7f53359aa640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53300b88f0 0x7f53300ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.276+0000 7f53361ab640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f53300d0cc0 0x7f53300ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.276+0000 7f53361ab640 1 -- 172.21.15.16:0/1834809324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53300b88f0 msgr2=0x7f53300ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.276+0000 7f53361ab640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53300b88f0 0x7f53300ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.276+0000 7f53361ab640 1 -- 172.21.15.16:0/1834809324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53300cc2b0 con 0x7f533015e510 2024-09-12T01:34:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.276+0000 7f53361ab640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533015e510 0x7f53300cdcd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5320009d90 tx=0x7f5320004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.276+0000 7f532e7fc640 1 -- 172.21.15.16:0/1834809324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320040070 con 0x7f533015e510 2024-09-12T01:34:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.276+0000 7f532e7fc640 1 -- 172.21.15.16:0/1834809324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5320031c90 con 0x7f533015e510 2024-09-12T01:34:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.276+0000 7f5337c0e640 1 -- 172.21.15.16:0/1834809324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533015f9d0 con 0x7f533015e510 2024-09-12T01:34:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.276+0000 7f532e7fc640 1 -- 172.21.15.16:0/1834809324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532003c640 con 0x7f533015e510 2024-09-12T01:34:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.276+0000 7f5337c0e640 1 -- 172.21.15.16:0/1834809324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f533015ff90 con 0x7f533015e510 2024-09-12T01:34:16.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.277+0000 7f5337c0e640 1 -- 172.21.15.16:0/1834809324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f533015f7c0 con 0x7f533015e510 2024-09-12T01:34:16.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.278+0000 7f532e7fc640 1 -- 172.21.15.16:0/1834809324 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5320039040 con 0x7f533015e510 2024-09-12T01:34:16.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.278+0000 7f532e7fc640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f531c06be30 0x7f531c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.278+0000 7f532e7fc640 1 -- 172.21.15.16:0/1834809324 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f53200ac740 con 0x7f533015e510 2024-09-12T01:34:16.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.278+0000 7f53359aa640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f531c06be30 0x7f531c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.279+0000 7f53359aa640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f531c06be30 0x7f531c06e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5328000f60 tx=0x7f5328009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:16.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.282+0000 7f532e7fc640 1 -- 172.21.15.16:0/1834809324 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f532007a9d0 con 0x7f533015e510 2024-09-12T01:34:16.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.495+0000 7f5337c0e640 1 -- 172.21.15.16:0/1834809324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53300d1120 con 0x7f533015e510 2024-09-12T01:34:16.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.496+0000 7f532e7fc640 1 -- 172.21.15.16:0/1834809324 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5214 (secure 0 0 0) 0x7f532007ecd0 con 0x7f533015e510 2024-09-12T01:34:16.496 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:16.496 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-12T01:34:10:019140+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/842803487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":842803487},{"type":"v1","addr":"172.21.15.33:6833","nonce":842803487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/70052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":70052721},{"type":"v1","addr":"172.21.15.33:6835","nonce":70052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2095784417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2095784417},{"type":"v1","addr":"172.21.15.16:6835","nonce":2095784417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:34:06.918906+0000","modified":"2024-09-12T01:34:09.376867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4932":{"gid":4932,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/4214019228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4214019228},{"type":"v1","addr":"172.21.15.16:6837","nonce":4214019228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4932,"qdb_cluster":[4932]},"id":2}]} 2024-09-12T01:34:16.496 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-12T01:34:16.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.498+0000 7f5337c0e640 1 -- 172.21.15.16:0/1834809324 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f531c06be30 msgr2=0x7f531c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.498+0000 7f5337c0e640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f531c06be30 0x7f531c06e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5328000f60 tx=0x7f5328009040 comp rx=0 tx=0).stop 2024-09-12T01:34:16.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.498+0000 7f5337c0e640 1 -- 172.21.15.16:0/1834809324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533015e510 msgr2=0x7f53300cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.498+0000 7f5337c0e640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533015e510 0x7f53300cdcd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5320009d90 tx=0x7f5320004a40 comp rx=0 tx=0).stop 2024-09-12T01:34:16.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.498+0000 7f5337c0e640 1 -- 172.21.15.16:0/1834809324 shutdown_connections 2024-09-12T01:34:16.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.498+0000 7f5337c0e640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f531c06be30 0x7f531c06e2f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.498+0000 7f5337c0e640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533015e510 0x7f53300cdcd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.498+0000 7f5337c0e640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f53300d0cc0 0x7f53300ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.498+0000 7f5337c0e640 1 --2- 172.21.15.16:0/1834809324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53300b88f0 0x7f53300ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.498+0000 7f5337c0e640 1 -- 172.21.15.16:0/1834809324 >> 172.21.15.16:0/1834809324 conn(0x7f533006d1a0 msgr2=0x7f53300cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:16.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.498+0000 7f5337c0e640 1 -- 172.21.15.16:0/1834809324 shutdown_connections 2024-09-12T01:34:16.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:16.498+0000 7f5337c0e640 1 -- 172.21.15.16:0/1834809324 wait complete. 2024-09-12T01:34:16.911 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T01:34:06.918906+0000', 'modified': '2024-09-12T01:34:09.376867+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4932}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4932': {'gid': 4932, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6837/4214019228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6836', 'nonce': 4214019228}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 4214019228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4932, 'qdb_cluster': [4932]} 2024-09-12T01:34:16.911 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T01:34:16.912 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-12T01:34:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.018+0000 7fecb3e2c640 1 Processor -- start 2024-09-12T01:34:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.019+0000 7fecb3e2c640 1 -- start start 2024-09-12T01:34:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.019+0000 7fecb3e2c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fecac158920 0x7fecac0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.019+0000 7fecb3e2c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecac0b4c30 con 0x7fecac0cf440 2024-09-12T01:34:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.019+0000 7fecb3e2c640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecac162090 con 0x7fecac06e630 2024-09-12T01:34:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.019+0000 7fecb3e2c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fecac162370 con 0x7fecac158920 2024-09-12T01:34:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.020+0000 7fecb23c9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fecac0cf440 0x7fecac15eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49254/0 (socket says 172.21.15.16:49254) 2024-09-12T01:34:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.020+0000 7fecb23c9640 1 -- 172.21.15.16:0/1353787914 learned_addr learned my addr 172.21.15.16:0/1353787914 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.020+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/1353787914 >> v2:172.21.15.16:3300/0 conn(0x7fecac158920 0x7fecac0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.020+0000 7fecb0bc6640 1 -- 172.21.15.16:0/1353787914 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3093930497 0 0) 0x7fecac0b4c30 con 0x7fecac0cf440 2024-09-12T01:34:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.020+0000 7fecb0bc6640 1 -- 172.21.15.16:0/1353787914 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec94003520 con 0x7fecac0cf440 2024-09-12T01:34:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.020+0000 7fecb1bc8640 1 -- 172.21.15.16:0/1353787914 >> v1:172.21.15.33:6789/0 conn(0x7fecac06e630 legacy=0x7fecac0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.020+0000 7fecb1bc8640 1 -- 172.21.15.16:0/1353787914 >> v1:172.21.15.16:6789/0 conn(0x7fecac0cf440 legacy=0x7fecac15eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.020+0000 7fecb1bc8640 1 -- 172.21.15.16:0/1353787914 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecac162650 con 0x7fecac158920 2024-09-12T01:34:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.021+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/1353787914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecac158920 0x7fecac0b4650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feca40097a0 tx=0x7feca402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2bec24de89f2b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.021+0000 7fecb0bc6640 1 -- 172.21.15.16:0/1353787914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4037070 con 0x7fecac158920 2024-09-12T01:34:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.021+0000 7fecb0bc6640 1 -- 172.21.15.16:0/1353787914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca4032440 con 0x7fecac158920 2024-09-12T01:34:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.021+0000 7fecb0bc6640 1 -- 172.21.15.16:0/1353787914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4032810 con 0x7fecac158920 2024-09-12T01:34:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.021+0000 7fecb3e2c640 1 -- 172.21.15.16:0/1353787914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecac158920 msgr2=0x7fecac0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.021+0000 7fecb3e2c640 1 --2- 172.21.15.16:0/1353787914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecac158920 0x7fecac0b4650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feca40097a0 tx=0x7feca402a1a0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.021+0000 7fecb3e2c640 1 -- 172.21.15.16:0/1353787914 shutdown_connections 2024-09-12T01:34:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.021+0000 7fecb3e2c640 1 --2- 172.21.15.16:0/1353787914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecac158920 0x7fecac0b4650 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.021+0000 7fecb3e2c640 1 -- 172.21.15.16:0/1353787914 >> 172.21.15.16:0/1353787914 conn(0x7fecac06feb0 msgr2=0x7fecac0702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.022+0000 7fecb3e2c640 1 -- 172.21.15.16:0/1353787914 shutdown_connections 2024-09-12T01:34:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.022+0000 7fecb3e2c640 1 -- 172.21.15.16:0/1353787914 wait complete. 2024-09-12T01:34:17.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.023+0000 7fecb3e2c640 1 Processor -- start 2024-09-12T01:34:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.023+0000 7fecb3e2c640 1 -- start start 2024-09-12T01:34:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.023+0000 7fecb3e2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecac06e630 0x7fecac0c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.023+0000 7fecb3e2c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac0cf440 0x7fecac156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.023+0000 7fecb3e2c640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fecac158920 0x7fecac0cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.023+0000 7fecb3e2c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fecac06eac0 con 0x7fecac06e630 2024-09-12T01:34:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.023+0000 7fecb3e2c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fecac162e50 con 0x7fecac0cf440 2024-09-12T01:34:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.023+0000 7fecb3e2c640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fecac06ec40 con 0x7fecac158920 2024-09-12T01:34:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecb1bc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecac06e630 0x7fecac0c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecb1bc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecac06e630 0x7fecac0c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45994/0 (socket says 172.21.15.16:45994) 2024-09-12T01:34:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecb1bc8640 1 -- 172.21.15.16:0/3023828778 learned_addr learned my addr 172.21.15.16:0/3023828778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecb13c7640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac0cf440 0x7fecac156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecb23c9640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fecac158920 0x7fecac0cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecb1bc8640 1 -- 172.21.15.16:0/3023828778 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fecac158920 msgr2=0x7fecac0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fecac158920 0x7fecac0cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecb1bc8640 1 -- 172.21.15.16:0/3023828778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac0cf440 msgr2=0x7fecac156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac0cf440 0x7fecac156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecb1bc8640 1 -- 172.21.15.16:0/3023828778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecac1e9e80 con 0x7fecac06e630 2024-09-12T01:34:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecac06e630 0x7fecac0c9ce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feca40098c0 tx=0x7feca40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecaa7fc640 1 -- 172.21.15.16:0/3023828778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4037040 con 0x7fecac06e630 2024-09-12T01:34:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecaa7fc640 1 -- 172.21.15.16:0/3023828778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca4030500 con 0x7fecac06e630 2024-09-12T01:34:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecaa7fc640 1 -- 172.21.15.16:0/3023828778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca40308d0 con 0x7fecac06e630 2024-09-12T01:34:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecb3e2c640 1 -- 172.21.15.16:0/3023828778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecac1ea160 con 0x7fecac06e630 2024-09-12T01:34:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.024+0000 7fecb3e2c640 1 -- 172.21.15.16:0/3023828778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecac1ea720 con 0x7fecac06e630 2024-09-12T01:34:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.026+0000 7fecaa7fc640 1 -- 172.21.15.16:0/3023828778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feca4044020 con 0x7fecac06e630 2024-09-12T01:34:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.027+0000 7fecaa7fc640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fec9806be30 0x7fec9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.027+0000 7fecb13c7640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fec9806be30 0x7fec9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.027+0000 7fecaa7fc640 1 -- 172.21.15.16:0/3023828778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7feca40ab1a0 con 0x7fecac06e630 2024-09-12T01:34:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.027+0000 7fecb13c7640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fec9806be30 0x7fec9806e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fecac0c9010 tx=0x7feca0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.028+0000 7fecb6bbf740 1 Processor -- start 2024-09-12T01:34:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.028+0000 7fecb6bbf740 1 -- start start 2024-09-12T01:34:17.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.028+0000 7fecb6bbf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3de7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.028+0000 7fecb6bbf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3d9fa80 0x564de3da26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.028+0000 7fecb6bbf740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 0x564de3d9c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.028+0000 7fecb6bbf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564de3d04520 con 0x564de3da22c0 2024-09-12T01:34:17.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.028+0000 7fecb6bbf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564de3cac3e0 con 0x564de3d9fa80 2024-09-12T01:34:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.028+0000 7fecb6bbf740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x564de3d004e0 con 0x564de3da3a50 2024-09-12T01:34:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.028+0000 7fecb1bc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3de7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.029+0000 7fecb1bc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3de7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45996/0 (socket says 172.21.15.16:45996) 2024-09-12T01:34:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.029+0000 7fecb1bc8640 1 -- 172.21.15.16:0/4130139706 learned_addr learned my addr 172.21.15.16:0/4130139706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.029+0000 7fecb23c9640 1 --2- 172.21.15.16:0/4130139706 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 0x564de3d9c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.029+0000 7fecb13c7640 1 --2- 172.21.15.16:0/4130139706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3d9fa80 0x564de3da26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.029+0000 7fecb1bc8640 1 -- 172.21.15.16:0/4130139706 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 msgr2=0x564de3d9c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.029+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/4130139706 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 0x564de3d9c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.029+0000 7fecb1bc8640 1 -- 172.21.15.16:0/4130139706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3d9fa80 msgr2=0x564de3da26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.029+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/4130139706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3d9fa80 0x564de3da26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.029+0000 7fecb1bc8640 1 -- 172.21.15.16:0/4130139706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de3cc72f0 con 0x564de3da22c0 2024-09-12T01:34:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.029+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/4130139706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3de7a60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7feca4035040 tx=0x7feca4078d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cb9c2ed3caadc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.029+0000 7fec937fe640 1 -- 172.21.15.16:0/4130139706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4081e40 con 0x564de3da22c0 2024-09-12T01:34:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.029+0000 7fec937fe640 1 -- 172.21.15.16:0/4130139706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca4077610 con 0x564de3da22c0 2024-09-12T01:34:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.029+0000 7fec937fe640 1 -- 172.21.15.16:0/4130139706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4075980 con 0x564de3da22c0 2024-09-12T01:34:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.030+0000 7fecb6bbf740 1 -- 172.21.15.16:0/4130139706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 msgr2=0x564de3de7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.030+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/4130139706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3de7a60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7feca4035040 tx=0x7feca4078d30 comp rx=0 tx=0).stop 2024-09-12T01:34:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.030+0000 7fecb6bbf740 1 -- 172.21.15.16:0/4130139706 shutdown_connections 2024-09-12T01:34:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.030+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/4130139706 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 0x564de3d9c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.030+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/4130139706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3d9fa80 0x564de3da26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.030+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/4130139706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3de7a60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.030+0000 7fecb6bbf740 1 -- 172.21.15.16:0/4130139706 >> 172.21.15.16:0/4130139706 conn(0x564de3d9ea80 msgr2=0x564de3de6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.030+0000 7fecb6bbf740 1 -- 172.21.15.16:0/4130139706 shutdown_connections 2024-09-12T01:34:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.030+0000 7fecb6bbf740 1 -- 172.21.15.16:0/4130139706 wait complete. 2024-09-12T01:34:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.030+0000 7fecb6bbf740 1 -- start start 2024-09-12T01:34:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.031+0000 7fecb6bbf740 1 Processor -- start 2024-09-12T01:34:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.031+0000 7fecb6bbf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3db7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.031+0000 7fecb6bbf740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 0x564de3dc9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.031+0000 7fecb6bbf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3dc67f0 0x564de3de1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.031+0000 7fecb6bbf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564de3d9df20 con 0x564de3da22c0 2024-09-12T01:34:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.031+0000 7fecb6bbf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564de3c8d450 con 0x564de3dc67f0 2024-09-12T01:34:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.031+0000 7fecb6bbf740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x564de3de8290 con 0x564de3da3a50 2024-09-12T01:34:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.031+0000 7fecb6bbf740 10 client.0 _ll_register_callbacks cb 0x564de3a16ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb1bc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3db7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb1bc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3db7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46008/0 (socket says 172.21.15.16:46008) 2024-09-12T01:34:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb1bc8640 1 -- 172.21.15.16:0/2663327520 learned_addr learned my addr 172.21.15.16:0/2663327520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb13c7640 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 0x564de3dc9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb23c9640 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3dc67f0 0x564de3de1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb1bc8640 1 -- 172.21.15.16:0/2663327520 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 msgr2=0x564de3dc9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 0x564de3dc9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb1bc8640 1 -- 172.21.15.16:0/2663327520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3dc67f0 msgr2=0x564de3de1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3dc67f0 0x564de3de1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb1bc8640 1 -- 172.21.15.16:0/2663327520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de3a172d0 con 0x564de3da22c0 2024-09-12T01:34:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb23c9640 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3dc67f0 0x564de3de1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3db7ac0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7feca40794d0 tx=0x7feca4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d94369a04edb2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fec90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:34:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fec90ff9640 1 -- 172.21.15.16:0/2663327520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4081e40 con 0x564de3da22c0 2024-09-12T01:34:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fec90ff9640 1 -- 172.21.15.16:0/2663327520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7feca40819b0 con 0x564de3da22c0 2024-09-12T01:34:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2663327520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca4077610 con 0x564de3da22c0 2024-09-12T01:34:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fec90ff9640 1 -- 172.21.15.16:0/2663327520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7feca4081bb0 con 0x564de3da22c0 2024-09-12T01:34:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb6bbf740 10 client.5121 fetch_fsmap learned FSMap version 16 2024-09-12T01:34:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2663327520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x564de3ae16b0 con 0x564de3da22c0 2024-09-12T01:34:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.032+0000 7fec90ff9640 1 -- 172.21.15.16:0/2663327520 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca4077610 con 0x564de3da22c0 2024-09-12T01:34:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.033+0000 7fec90ff9640 1 -- 172.21.15.16:0/2663327520 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7feca403a070 con 0x564de3da22c0 2024-09-12T01:34:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.033+0000 7fecb6bbf740 10 client.5121 fetch_fsmap finished waiting for FSMap version 16 2024-09-12T01:34:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.033+0000 7fecb6bbf740 10 client.5121 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T01:34:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.033+0000 7fecb6bbf740 20 client.5121 populate_metadata read hostname 'smithi016' 2024-09-12T01:34:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.033+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.16:6836/4214019228,v1:172.21.15.16:6837/4214019228] conn(0x564de3dcc730 0x564de3de2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.033+0000 7fecb6bbf740 4 client.5121 mds_command: new command op to 4932 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:34:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.033+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2663327520 --> [v2:172.21.15.16:6836/4214019228,v1:172.21.15.16:6837/4214019228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564de3d11c50 con 0x564de3dcc730 2024-09-12T01:34:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.033+0000 7fecb13c7640 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.16:6836/4214019228,v1:172.21.15.16:6837/4214019228] conn(0x564de3dcc730 0x564de3de2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.033+0000 7fec90ff9640 10 client.5121 ms_handle_connect on v2:172.21.15.16:6836/4214019228 2024-09-12T01:34:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.033+0000 7fecb13c7640 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.16:6836/4214019228,v1:172.21.15.16:6837/4214019228] conn(0x564de3dcc730 0x564de3de2bf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac4b14323949b60 server_cookie=6f81669772ebfcc in_seq=0 out_seq=0 2024-09-12T01:34:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fec90ff9640 1 -- 172.21.15.16:0/2663327520 <== mds.0 v2:172.21.15.16:6836/4214019228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564de3d11c50 con 0x564de3dcc730 2024-09-12T01:34:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fec90ff9640 10 client.5121 handle_command_reply: tid=0 2024-09-12T01:34:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fecb6bbf740 1 client.5121 shutdown 2024-09-12T01:34:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2663327520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 msgr2=0x564de3db7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3db7ac0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7feca40794d0 tx=0x7feca4009190 comp rx=0 tx=0).stop 2024-09-12T01:34:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2663327520 shutdown_connections 2024-09-12T01:34:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.16:6836/4214019228,v1:172.21.15.16:6837/4214019228] conn(0x564de3dcc730 0x564de3de2bf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3dc67f0 0x564de3de1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 0x564de3dc9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fec90ff9640 0 client.5121 ms_handle_reset on v2:172.21.15.16:6836/4214019228 2024-09-12T01:34:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/2663327520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3db7ac0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2663327520 >> 172.21.15.16:0/2663327520 conn(0x564de3d9fa80 msgr2=0x564de3de6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2663327520 shutdown_connections 2024-09-12T01:34:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2663327520 wait complete. 2024-09-12T01:34:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.034+0000 7fecb6bbf740 20 client.5121 trim_cache size 0 max 16384 2024-09-12T01:34:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.054+0000 7fecb6bbf740 1 Processor -- start 2024-09-12T01:34:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.054+0000 7fecb6bbf740 1 -- start start 2024-09-12T01:34:17.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb6bbf740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da22c0 0x564de3de6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb6bbf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3da3a50 0x564de3de72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb6bbf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3dc67f0 0x564de3dc81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb6bbf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1fff40 con 0x564de3dc67f0 2024-09-12T01:34:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb6bbf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fecac0b6a60 con 0x564de3da3a50 2024-09-12T01:34:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb6bbf740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fecac0b70b0 con 0x564de3da22c0 2024-09-12T01:34:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb23c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3dc67f0 0x564de3dc81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb23c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3dc67f0 0x564de3dc81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46026/0 (socket says 172.21.15.16:46026) 2024-09-12T01:34:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb23c9640 1 -- 172.21.15.16:0/1508750739 learned_addr learned my addr 172.21.15.16:0/1508750739 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/1508750739 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da22c0 0x564de3de6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb13c7640 1 --2- 172.21.15.16:0/1508750739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3da3a50 0x564de3de72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb23c9640 1 -- 172.21.15.16:0/1508750739 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da22c0 msgr2=0x564de3de6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb23c9640 1 --2- 172.21.15.16:0/1508750739 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da22c0 0x564de3de6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb23c9640 1 -- 172.21.15.16:0/1508750739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3da3a50 msgr2=0x564de3de72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb23c9640 1 --2- 172.21.15.16:0/1508750739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3da3a50 0x564de3de72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb23c9640 1 -- 172.21.15.16:0/1508750739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de3d988b0 con 0x564de3dc67f0 2024-09-12T01:34:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.055+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/1508750739 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da22c0 0x564de3de6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.056+0000 7fecb23c9640 1 --2- 172.21.15.16:0/1508750739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3dc67f0 0x564de3dc81f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fec9c002000 tx=0x7fec9c001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e639a6e08d12dcaf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.056+0000 7fec92ffd640 1 -- 172.21.15.16:0/1508750739 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c01a070 con 0x564de3dc67f0 2024-09-12T01:34:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.056+0000 7fec92ffd640 1 -- 172.21.15.16:0/1508750739 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec9c007ab0 con 0x564de3dc67f0 2024-09-12T01:34:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.056+0000 7fec92ffd640 1 -- 172.21.15.16:0/1508750739 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c016830 con 0x564de3dc67f0 2024-09-12T01:34:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.056+0000 7fecb6bbf740 1 -- 172.21.15.16:0/1508750739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3dc67f0 msgr2=0x564de3dc81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.056+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/1508750739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3dc67f0 0x564de3dc81f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fec9c002000 tx=0x7fec9c001a10 comp rx=0 tx=0).stop 2024-09-12T01:34:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.056+0000 7fecb6bbf740 1 -- 172.21.15.16:0/1508750739 shutdown_connections 2024-09-12T01:34:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.056+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/1508750739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3dc67f0 0x564de3dc81f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.056+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/1508750739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3da3a50 0x564de3de72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.057+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/1508750739 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da22c0 0x564de3de6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.057+0000 7fecb6bbf740 1 -- 172.21.15.16:0/1508750739 >> 172.21.15.16:0/1508750739 conn(0x564de3d9fa80 msgr2=0x564de3de1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:17.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.057+0000 7fecb6bbf740 1 -- 172.21.15.16:0/1508750739 shutdown_connections 2024-09-12T01:34:17.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.057+0000 7fecb6bbf740 1 -- 172.21.15.16:0/1508750739 wait complete. 2024-09-12T01:34:17.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.057+0000 7fecb6bbf740 1 -- start start 2024-09-12T01:34:17.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.057+0000 7fecb6bbf740 1 Processor -- start 2024-09-12T01:34:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.058+0000 7fecb6bbf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3e0a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.058+0000 7fecb6bbf740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 0x564de3e68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.058+0000 7fecb6bbf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3dc67f0 0x564de3dea630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.058+0000 7fecb6bbf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564de3d9df20 con 0x564de3da22c0 2024-09-12T01:34:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.058+0000 7fecb6bbf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564de3de8290 con 0x564de3dc67f0 2024-09-12T01:34:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.058+0000 7fecb6bbf740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x564de3c8d450 con 0x564de3da3a50 2024-09-12T01:34:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.058+0000 7fecb6bbf740 10 client.0 _ll_register_callbacks cb 0x7feca4090070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.058+0000 7fecb1bc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3e0a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.058+0000 7fecb1bc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3e0a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46038/0 (socket says 172.21.15.16:46038) 2024-09-12T01:34:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.058+0000 7fecb1bc8640 1 -- 172.21.15.16:0/2072004397 learned_addr learned my addr 172.21.15.16:0/2072004397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.058+0000 7fecb13c7640 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 0x564de3e68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.058+0000 7fecb23c9640 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3dc67f0 0x564de3dea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb1bc8640 1 -- 172.21.15.16:0/2072004397 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 msgr2=0x564de3e68760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 0x564de3e68760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb1bc8640 1 -- 172.21.15.16:0/2072004397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3dc67f0 msgr2=0x564de3dea630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3dc67f0 0x564de3dea630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb1bc8640 1 -- 172.21.15.16:0/2072004397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de3dca240 con 0x564de3da22c0 2024-09-12T01:34:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb13c7640 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 0x564de3e68760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb1bc8640 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3e0a4a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feca4081910 tx=0x7feca40094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6921789a04655e99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fec90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:34:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fec90ff9640 1 -- 172.21.15.16:0/2072004397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4081e40 con 0x564de3da22c0 2024-09-12T01:34:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fec90ff9640 1 -- 172.21.15.16:0/2072004397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7feca4075980 con 0x564de3da22c0 2024-09-12T01:34:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2072004397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564de3d0bfa0 con 0x564de3da22c0 2024-09-12T01:34:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fec90ff9640 1 -- 172.21.15.16:0/2072004397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7feca4078970 con 0x564de3da22c0 2024-09-12T01:34:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb6bbf740 10 client.5127 fetch_fsmap learned FSMap version 16 2024-09-12T01:34:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2072004397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x564de3dca240 con 0x564de3da22c0 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fec90ff9640 1 -- 172.21.15.16:0/2072004397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564de3d0bfa0 con 0x564de3da22c0 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fec90ff9640 1 -- 172.21.15.16:0/2072004397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7feca403a050 con 0x564de3da22c0 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb6bbf740 10 client.5127 fetch_fsmap finished waiting for FSMap version 16 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb6bbf740 10 client.5127 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb6bbf740 20 client.5127 populate_metadata read hostname 'smithi016' 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.16:6836/4214019228,v1:172.21.15.16:6837/4214019228] conn(0x564de3e63f50 0x564de3e64330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb6bbf740 4 client.5127 mds_command: new command op to 4932 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.059+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2072004397 --> [v2:172.21.15.16:6836/4214019228,v1:172.21.15.16:6837/4214019228] -- command(tid 0: {"prefix": "status"}) -- 0x564de3c88b80 con 0x564de3e63f50 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fecb13c7640 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.16:6836/4214019228,v1:172.21.15.16:6837/4214019228] conn(0x564de3e63f50 0x564de3e64330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fec90ff9640 10 client.5127 ms_handle_connect on v2:172.21.15.16:6836/4214019228 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fecb13c7640 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.16:6836/4214019228,v1:172.21.15.16:6837/4214019228] conn(0x564de3e63f50 0x564de3e64330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1081afafa263e9d0 server_cookie=a26448bdc238b30a in_seq=0 out_seq=0 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fec90ff9640 1 -- 172.21.15.16:0/2072004397 <== mds.0 v2:172.21.15.16:6836/4214019228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x564de3c88b80 con 0x564de3e63f50 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fec90ff9640 10 client.5127 handle_command_reply: tid=0 2024-09-12T01:34:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fecb6bbf740 1 client.5127 shutdown 2024-09-12T01:34:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2072004397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 msgr2=0x564de3e0a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3e0a4a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feca4081910 tx=0x7feca40094b0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2072004397 shutdown_connections 2024-09-12T01:34:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.16:6836/4214019228,v1:172.21.15.16:6837/4214019228] conn(0x564de3e63f50 0x564de3e64330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de3dc67f0 0x564de3dea630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x564de3da3a50 0x564de3e68760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fec90ff9640 0 client.5127 ms_handle_reset on v2:172.21.15.16:6836/4214019228 2024-09-12T01:34:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fecb6bbf740 1 --2- 172.21.15.16:0/2072004397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564de3da22c0 0x564de3e0a4a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2072004397 >> 172.21.15.16:0/2072004397 conn(0x564de3d9fa80 msgr2=0x564de3de6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2072004397 shutdown_connections 2024-09-12T01:34:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.060+0000 7fecb6bbf740 1 -- 172.21.15.16:0/2072004397 wait complete. 2024-09-12T01:34:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.061+0000 7fecb6bbf740 20 client.5127 trim_cache size 0 max 16384 2024-09-12T01:34:17.067 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:34:17.067 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "cb5903e4-2f7f-485b-9d92-173a6afaa2f8", 2024-09-12T01:34:17.067 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-12T01:34:17.067 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4932, 2024-09-12T01:34:17.067 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T01:34:17.067 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T01:34:17.067 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T01:34:17.067 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.5084197970000002, 2024-09-12T01:34:17.067 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 16, 2024-09-12T01:34:17.067 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 40, 2024-09-12T01:34:17.068 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 40, 2024-09-12T01:34:17.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 16.361885224000002 2024-09-12T01:34:17.068 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:34:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.061+0000 7fecb3e2c640 1 -- 172.21.15.16:0/3023828778 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fec9806be30 msgr2=0x7fec9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.061+0000 7fecb3e2c640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fec9806be30 0x7fec9806e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fecac0c9010 tx=0x7feca0009040 comp rx=0 tx=0).stop 2024-09-12T01:34:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.061+0000 7fecb3e2c640 1 -- 172.21.15.16:0/3023828778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecac06e630 msgr2=0x7fecac0c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.061+0000 7fecb3e2c640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecac06e630 0x7fecac0c9ce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feca40098c0 tx=0x7feca40046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.061+0000 7fecb3e2c640 1 -- 172.21.15.16:0/3023828778 shutdown_connections 2024-09-12T01:34:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.061+0000 7fecb3e2c640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fec9806be30 0x7fec9806e2f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.061+0000 7fecb3e2c640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fecac158920 0x7fecac0cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.061+0000 7fecb3e2c640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac0cf440 0x7fecac156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.061+0000 7fecb3e2c640 1 --2- 172.21.15.16:0/3023828778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecac06e630 0x7fecac0c9ce0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.061+0000 7fecb3e2c640 1 -- 172.21.15.16:0/3023828778 >> 172.21.15.16:0/3023828778 conn(0x7fecac06feb0 msgr2=0x7fecac0c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.061+0000 7fecb3e2c640 1 -- 172.21.15.16:0/3023828778 shutdown_connections 2024-09-12T01:34:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.061+0000 7fecb3e2c640 1 -- 172.21.15.16:0/3023828778 wait complete. 2024-09-12T01:34:17.072 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T01:34:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.175+0000 7f35607f1640 1 Processor -- start 2024-09-12T01:34:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.176+0000 7f35607f1640 1 -- start start 2024-09-12T01:34:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.176+0000 7f35607f1640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f355815e510 0x7f35580ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.176+0000 7f35607f1640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f35580d13d0 0x7f35580b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.177+0000 7f35607f1640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35580d3430 con 0x7f35580cf040 2024-09-12T01:34:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.177+0000 7f35607f1640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f35580d36d0 con 0x7f355815e510 2024-09-12T01:34:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.177+0000 7f35607f1640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f35580d39b0 con 0x7f35580d13d0 2024-09-12T01:34:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.177+0000 7f355ed8e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f35580d13d0 0x7f35580b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.177+0000 7f355e58d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f355815e510 0x7f35580ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.177+0000 7f355ed8e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f35580d13d0 0x7f35580b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:33450/0 (socket says 172.21.15.16:33450) 2024-09-12T01:34:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.177+0000 7f355ed8e640 1 -- 172.21.15.16:0/2972908713 learned_addr learned my addr 172.21.15.16:0/2972908713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.177+0000 7f355ed8e640 1 -- 172.21.15.16:0/2972908713 >> v2:172.21.15.33:3300/0 conn(0x7f355815e510 msgr2=0x7f35580ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.177+0000 7f355ed8e640 1 --2- 172.21.15.16:0/2972908713 >> v2:172.21.15.33:3300/0 conn(0x7f355815e510 0x7f35580ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.177+0000 7f355ed8e640 1 -- 172.21.15.16:0/2972908713 >> v1:172.21.15.33:6789/0 conn(0x7f35580cf040 legacy=0x7f35580d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.177+0000 7f355ed8e640 1 -- 172.21.15.16:0/2972908713 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35580d3c90 con 0x7f35580d13d0 2024-09-12T01:34:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.178+0000 7f355ed8e640 1 --2- 172.21.15.16:0/2972908713 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f35580d13d0 0x7f35580b78e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3550009e50 tx=0x7f355002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46e726df47b6a8c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.178+0000 7f355d58b640 1 -- 172.21.15.16:0/2972908713 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550036070 con 0x7f35580d13d0 2024-09-12T01:34:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.178+0000 7f355d58b640 1 -- 172.21.15.16:0/2972908713 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3550031600 con 0x7f35580d13d0 2024-09-12T01:34:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.178+0000 7f355d58b640 1 -- 172.21.15.16:0/2972908713 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35500319d0 con 0x7f35580d13d0 2024-09-12T01:34:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.179+0000 7f35607f1640 1 -- 172.21.15.16:0/2972908713 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f35580d13d0 msgr2=0x7f35580b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.179+0000 7f35607f1640 1 --2- 172.21.15.16:0/2972908713 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f35580d13d0 0x7f35580b78e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3550009e50 tx=0x7f355002a680 comp rx=0 tx=0).stop 2024-09-12T01:34:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.179+0000 7f35607f1640 1 -- 172.21.15.16:0/2972908713 shutdown_connections 2024-09-12T01:34:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.179+0000 7f35607f1640 1 --2- 172.21.15.16:0/2972908713 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f35580d13d0 0x7f35580b78e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.179+0000 7f35607f1640 1 --2- 172.21.15.16:0/2972908713 >> v2:172.21.15.33:3300/0 conn(0x7f355815e510 0x7f35580ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.179+0000 7f35607f1640 1 -- 172.21.15.16:0/2972908713 >> 172.21.15.16:0/2972908713 conn(0x7f355806d1a0 msgr2=0x7f3558073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.179+0000 7f35607f1640 1 -- 172.21.15.16:0/2972908713 shutdown_connections 2024-09-12T01:34:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.179+0000 7f35607f1640 1 -- 172.21.15.16:0/2972908713 wait complete. 2024-09-12T01:34:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.180+0000 7f35607f1640 1 Processor -- start 2024-09-12T01:34:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.180+0000 7f35607f1640 1 -- start start 2024-09-12T01:34:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f35607f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35580cf040 0x7f35581f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f35607f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35580d13d0 0x7f35580b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f35607f1640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f355815e510 0x7f3558160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f35607f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f355815e8f0 con 0x7f35580d13d0 2024-09-12T01:34:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f35607f1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35580d22a0 con 0x7f35580cf040 2024-09-12T01:34:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f35607f1640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f355815ea70 con 0x7f355815e510 2024-09-12T01:34:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f355dd8c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35580d13d0 0x7f35580b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f355dd8c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35580d13d0 0x7f35580b3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46044/0 (socket says 172.21.15.16:46044) 2024-09-12T01:34:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f355dd8c640 1 -- 172.21.15.16:0/2977987188 learned_addr learned my addr 172.21.15.16:0/2977987188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f355ed8e640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f355815e510 0x7f3558160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f355e58d640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35580cf040 0x7f35581f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f355dd8c640 1 -- 172.21.15.16:0/2977987188 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f355815e510 msgr2=0x7f3558160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f355dd8c640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f355815e510 0x7f3558160490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f355dd8c640 1 -- 172.21.15.16:0/2977987188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35580cf040 msgr2=0x7f35581f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f355dd8c640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35580cf040 0x7f35581f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f355dd8c640 1 -- 172.21.15.16:0/2977987188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35581609d0 con 0x7f35580d13d0 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.181+0000 7f355dd8c640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35580d13d0 0x7f35580b3f10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f354800c400 tx=0x7f354800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.182+0000 7f354effd640 1 -- 172.21.15.16:0/2977987188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35480052d0 con 0x7f35580d13d0 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.182+0000 7f354effd640 1 -- 172.21.15.16:0/2977987188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3548005490 con 0x7f35580d13d0 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.182+0000 7f354effd640 1 -- 172.21.15.16:0/2977987188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548007260 con 0x7f35580d13d0 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.182+0000 7f35607f1640 1 -- 172.21.15.16:0/2977987188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35580b4680 con 0x7f35580d13d0 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.182+0000 7f35607f1640 1 -- 172.21.15.16:0/2977987188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35580b4c40 con 0x7f35580d13d0 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.183+0000 7f35607f1640 1 -- 172.21.15.16:0/2977987188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35580ceb00 con 0x7f35580d13d0 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.183+0000 7f354effd640 1 -- 172.21.15.16:0/2977987188 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3548007400 con 0x7f35580d13d0 2024-09-12T01:34:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.184+0000 7f354effd640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f353c06be60 0x7f353c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.184+0000 7f354effd640 1 -- 172.21.15.16:0/2977987188 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f354808a430 con 0x7f35580d13d0 2024-09-12T01:34:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.184+0000 7f355e58d640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f353c06be60 0x7f353c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.184+0000 7f355e58d640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f353c06be60 0x7f353c06e320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3554009aa0 tx=0x7f3554009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.188+0000 7f354effd640 1 -- 172.21.15.16:0/2977987188 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f35480585f0 con 0x7f35580d13d0 2024-09-12T01:34:17.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.398+0000 7f35607f1640 1 -- 172.21.15.16:0/2977987188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3558074ad0 con 0x7f35580d13d0 2024-09-12T01:34:17.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.398+0000 7f354effd640 1 -- 172.21.15.16:0/2977987188 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f354805c8f0 con 0x7f35580d13d0 2024-09-12T01:34:17.398 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T01:34:17.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.400+0000 7f35607f1640 1 -- 172.21.15.16:0/2977987188 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f353c06be60 msgr2=0x7f353c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.400+0000 7f35607f1640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f353c06be60 0x7f353c06e320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3554009aa0 tx=0x7f3554009190 comp rx=0 tx=0).stop 2024-09-12T01:34:17.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.400+0000 7f35607f1640 1 -- 172.21.15.16:0/2977987188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35580d13d0 msgr2=0x7f35580b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:17.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.400+0000 7f35607f1640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35580d13d0 0x7f35580b3f10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f354800c400 tx=0x7f354800c8c0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.400+0000 7f35607f1640 1 -- 172.21.15.16:0/2977987188 shutdown_connections 2024-09-12T01:34:17.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.400+0000 7f35607f1640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f353c06be60 0x7f353c06e320 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.400+0000 7f35607f1640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f355815e510 0x7f3558160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.400+0000 7f35607f1640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35580d13d0 0x7f35580b3f10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.400+0000 7f35607f1640 1 --2- 172.21.15.16:0/2977987188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35580cf040 0x7f35581f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:17.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.400+0000 7f35607f1640 1 -- 172.21.15.16:0/2977987188 >> 172.21.15.16:0/2977987188 conn(0x7f355806d1a0 msgr2=0x7f3558159740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:17.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.401+0000 7f35607f1640 1 -- 172.21.15.16:0/2977987188 shutdown_connections 2024-09-12T01:34:17.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17.401+0000 7f35607f1640 1 -- 172.21.15.16:0/2977987188 wait complete. 2024-09-12T01:34:17.411 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T01:34:17.411 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T01:34:17.411 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-12T01:34:17.412 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T01:34:17.412 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T01:34:17.412 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T01:34:17.412 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T01:34:17.412 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T01:34:17.412 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T01:34:17.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:17.412 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-12T01:34:17.429 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T01:34:17.429 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T01:34:17.429 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T01:34:17.429 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:34:17.429 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-12T01:34:17.429 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:34:17.429 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:34:17.429 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:34:17.429 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-12T01:34:17.430 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:34:17.430 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:34:17.430 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-12T01:34:17.430 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T01:34:17.430 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:34:17.430 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-12T01:34:17.430 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T01:34:17.430 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42705sec preferred_lft 42705sec 2024-09-12T01:34:17.430 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-12T01:34:17.430 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:34:17.430 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:34:17.430 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:34:17.430 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-12T01:34:17.431 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T01:34:17.431 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 22:ee:75:26:5c:a8 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:34:17.431 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T01:34:17.431 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:34:17.431 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::2c48:d1ff:feb5:607b/64 scope link 2024-09-12T01:34:17.431 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:34:17.431 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T01:34:17.431 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether fe:94:45:31:d7:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:34:17.431 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::fc94:45ff:fe31:d756/64 scope link 2024-09-12T01:34:17.431 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:34:17.431 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T01:34:17.431 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 22:ee:75:26:5c:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T01:34:17.431 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::20ee:75ff:fe26:5ca8/64 scope link 2024-09-12T01:34:17.432 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:34:17.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:17.432 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T01:34:17.494 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-12T01:34:17.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:17.495 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-12T01:34:17.553 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-12T01:34:17.553 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T01:34:17.553 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T01:34:17.553 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T01:34:17.553 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T01:34:17.553 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:34:17.553 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T01:34:17.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T01:34:17.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T01:34:17.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T01:34:17.554 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T01:34:17.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:34:17.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:34:17.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:17.682 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-12T01:34:17.698 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T01:34:17.698 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T01:34:17.698 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T01:34:17.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:17.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-12T01:34:17.765 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-12T01:34:17.765 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:34:17.765 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T01:34:17.820 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T01:34:17.821 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:34:17.821 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T01:34:17.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-12T01:34:17.943 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:34:17.999 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-12T01:34:17.999 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-12T01:34:17.999 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-12T01:34:17.999 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-12T01:34:18.000 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-12T01:34:18.000 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-12T01:34:18.000 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-12T01:34:18.000 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-12T01:34:18.000 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-12T01:34:18.000 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-12T01:34:18.000 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-12T01:34:18.000 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-12T01:34:18.000 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:34:18.000 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-12T01:34:18.000 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-12T01:34:18.000 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-12T01:34:18.000 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-12T01:34:18.001 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-12T01:34:18.001 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-12T01:34:18.001 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:34:18.001 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:34:18.001 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:34:18.001 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:34:18.001 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-12T01:34:18.001 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-12T01:34:18.001 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T01:34:18.001 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T01:34:18.001 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T01:34:18.001 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T01:34:18.001 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-12T01:34:18.002 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:34:18.002 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:34:18.002 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T01:34:18.002 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T01:34:18.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:18.002 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-12T01:34:18.055 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T01:34:18.055 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-12T01:34:18.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-12T01:34:18.124 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:34:18.181 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-12T01:34:18.181 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-12T01:34:18.181 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-12T01:34:18.181 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-12T01:34:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-12T01:34:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-12T01:34:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-12T01:34:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-12T01:34:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-12T01:34:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-12T01:34:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-12T01:34:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-12T01:34:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:34:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-12T01:34:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-12T01:34:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T01:34:18.183 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-12T01:34:18.184 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:34:18.184 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:34:18.184 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T01:34:18.184 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T01:34:18.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:18.184 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-12T01:34:18.185 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-12T01:34:18.184+0000 7f7d57d50480 -1 init, newargv = 0x55e57ea49260 newargc=15 2024-09-12T01:34:18.185 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[48210]: starting ceph client 2024-09-12T01:34:18.188 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[48210]: starting fuse 2024-09-12T01:34:18.204 INFO:teuthology.orchestra.run.smithi016.stdout:43 2024-09-12T01:34:18.204 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-09-12T01:34:18.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-12T01:34:18.204 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-12T01:34:18.205 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:34:18.206 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-12T01:34:18.206 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T01:34:18.206 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T01:34:18.206 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T01:34:18.206 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-12T01:34:18.206 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-12T01:34:18.206 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-12T01:34:18.206 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T01:34:18.206 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-12T01:34:18.206 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-12T01:34:18.206 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-12T01:34:18.325 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.48210.asok 2024-09-12T01:34:18.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:34:18.335 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48210.asok 2024-09-12T01:34:18.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:18.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48210.asok status 2024-09-12T01:34:18.472 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:34:18.473 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-12T01:34:18.473 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T01:34:18.473 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:34:18.473 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-12T01:34:18.473 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-12T01:34:18.473 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T01:34:18.473 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "48210", 2024-09-12T01:34:18.473 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-12T01:34:18.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:18.473 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-12T01:34:18.473 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-12T01:34:18.474 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5136, 2024-09-12T01:34:18.474 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-12T01:34:18.474 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-12T01:34:18.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-12T01:34:18.474 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5136 2024-09-12T01:34:18.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:18.474 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-12T01:34:18.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 628061984 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 628061984 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5136 192.168.0.1:0/628061984", 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/628061984", 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 16, 2024-09-12T01:34:18.475 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 40, 2024-09-12T01:34:18.476 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-12T01:34:18.476 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-12T01:34:18.476 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-12T01:34:18.476 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:34:18.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:18.481 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-12T01:34:18.481 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T01:34:18.497 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-12T01:34:18.498 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T01:34:18.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:18.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T01:34:18.564 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-12T01:34:18.564 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-12T01:34:18.564 DEBUG:teuthology.orchestra.run.smithi016:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir") 2024-09-12T01:34:18.564 DEBUG:teuthology.orchestra.run.smithi016:> fd = os.open("/home/ubuntu/cephtest/mnt.0/testdir", os.O_RDONLY) 2024-09-12T01:34:18.564 DEBUG:teuthology.orchestra.run.smithi016:> os.fchmod(fd, 0o777) 2024-09-12T01:34:18.564 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(fd) 2024-09-12T01:34:18.564 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-12T01:34:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:34:18.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:34:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.829+0000 7fef71b13640 1 Processor -- start 2024-09-12T01:34:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.829+0000 7fef71b13640 1 -- start start 2024-09-12T01:34:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef71b13640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef6c15e8d0 0x7fef6c06dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef71b13640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fef6c0acdd0 0x7fef6c0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef71b13640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fef6c0cb3b0 0x7fef6c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef71b13640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fef6c06f530 con 0x7fef6c15e8d0 2024-09-12T01:34:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef71b13640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fef6c0cbbb0 con 0x7fef6c0cb3b0 2024-09-12T01:34:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef71b13640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fef6c0cbe90 con 0x7fef6c0acdd0 2024-09-12T01:34:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef6b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef6c15e8d0 0x7fef6c06dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef6b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef6c15e8d0 0x7fef6c06dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46054/0 (socket says 172.21.15.16:46054) 2024-09-12T01:34:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef6b7fe640 1 -- 172.21.15.16:0/2169046535 learned_addr learned my addr 172.21.15.16:0/2169046535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef6affd640 1 --2- 172.21.15.16:0/2169046535 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fef6c0acdd0 0x7fef6c0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef6bfff640 1 --2- 172.21.15.16:0/2169046535 >> v2:172.21.15.33:3300/0 conn(0x7fef6c0cb3b0 0x7fef6c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef6b7fe640 1 -- 172.21.15.16:0/2169046535 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fef6c0acdd0 msgr2=0x7fef6c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef6b7fe640 1 --2- 172.21.15.16:0/2169046535 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fef6c0acdd0 0x7fef6c0cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef6b7fe640 1 -- 172.21.15.16:0/2169046535 >> v2:172.21.15.33:3300/0 conn(0x7fef6c0cb3b0 msgr2=0x7fef6c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef6b7fe640 1 --2- 172.21.15.16:0/2169046535 >> v2:172.21.15.33:3300/0 conn(0x7fef6c0cb3b0 0x7fef6c0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef6b7fe640 1 -- 172.21.15.16:0/2169046535 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c0ca5f0 con 0x7fef6c15e8d0 2024-09-12T01:34:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.830+0000 7fef6bfff640 1 --2- 172.21.15.16:0/2169046535 >> v2:172.21.15.33:3300/0 conn(0x7fef6c0cb3b0 0x7fef6c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.831+0000 7fef6b7fe640 1 --2- 172.21.15.16:0/2169046535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c15e8d0 0x7fef6c06dd90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009870 tx=0x7fef5c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8554c266e9fcf96f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.831+0000 7fef6a7fc640 1 -- 172.21.15.16:0/2169046535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c037070 con 0x7fef6c15e8d0 2024-09-12T01:34:18.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.831+0000 7fef6a7fc640 1 -- 172.21.15.16:0/2169046535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5c032440 con 0x7fef6c15e8d0 2024-09-12T01:34:18.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.831+0000 7fef6a7fc640 1 -- 172.21.15.16:0/2169046535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c032810 con 0x7fef6c15e8d0 2024-09-12T01:34:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.832+0000 7fef71b13640 1 -- 172.21.15.16:0/2169046535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c15e8d0 msgr2=0x7fef6c06dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.832+0000 7fef71b13640 1 --2- 172.21.15.16:0/2169046535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c15e8d0 0x7fef6c06dd90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009870 tx=0x7fef5c02a200 comp rx=0 tx=0).stop 2024-09-12T01:34:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.832+0000 7fef71b13640 1 -- 172.21.15.16:0/2169046535 shutdown_connections 2024-09-12T01:34:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.832+0000 7fef71b13640 1 --2- 172.21.15.16:0/2169046535 >> v2:172.21.15.33:3300/0 conn(0x7fef6c0cb3b0 0x7fef6c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.832+0000 7fef71b13640 1 --2- 172.21.15.16:0/2169046535 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fef6c0acdd0 0x7fef6c0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.832+0000 7fef71b13640 1 --2- 172.21.15.16:0/2169046535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c15e8d0 0x7fef6c06dd90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.832+0000 7fef71b13640 1 -- 172.21.15.16:0/2169046535 >> 172.21.15.16:0/2169046535 conn(0x7fef6c0cffa0 msgr2=0x7fef6c0d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.832+0000 7fef71b13640 1 -- 172.21.15.16:0/2169046535 shutdown_connections 2024-09-12T01:34:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.832+0000 7fef71b13640 1 -- 172.21.15.16:0/2169046535 wait complete. 2024-09-12T01:34:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.833+0000 7fef71b13640 1 Processor -- start 2024-09-12T01:34:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.833+0000 7fef71b13640 1 -- start start 2024-09-12T01:34:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.834+0000 7fef71b13640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fef6c0acdd0 0x7fef6c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.834+0000 7fef71b13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c0cb3b0 0x7fef6c0c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.834+0000 7fef71b13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c15e8d0 0x7fef6c0b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.834+0000 7fef71b13640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c0cb840 con 0x7fef6c0cb3b0 2024-09-12T01:34:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.834+0000 7fef71b13640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c0cabb0 con 0x7fef6c15e8d0 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.834+0000 7fef71b13640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fef6c0ceda0 con 0x7fef6c0acdd0 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.834+0000 7fef6affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c0cb3b0 0x7fef6c0c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.834+0000 7fef6affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c0cb3b0 0x7fef6c0c3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46060/0 (socket says 172.21.15.16:46060) 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.834+0000 7fef6affd640 1 -- 172.21.15.16:0/1742998003 learned_addr learned my addr 172.21.15.16:0/1742998003 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.834+0000 7fef6b7fe640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fef6c0acdd0 0x7fef6c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.834+0000 7fef6bfff640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c15e8d0 0x7fef6c0b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.835+0000 7fef6affd640 1 -- 172.21.15.16:0/1742998003 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fef6c0acdd0 msgr2=0x7fef6c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.835+0000 7fef6affd640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fef6c0acdd0 0x7fef6c15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.835+0000 7fef6affd640 1 -- 172.21.15.16:0/1742998003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c15e8d0 msgr2=0x7fef6c0b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.835+0000 7fef6affd640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c15e8d0 0x7fef6c0b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.835+0000 7fef6affd640 1 -- 172.21.15.16:0/1742998003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c0c41b0 con 0x7fef6c0cb3b0 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.835+0000 7fef6bfff640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c15e8d0 0x7fef6c0b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.835+0000 7fef6affd640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c0cb3b0 0x7fef6c0c3c70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fef6000d830 tx=0x7fef6000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.835+0000 7fef53fff640 1 -- 172.21.15.16:0/1742998003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60018020 con 0x7fef6c0cb3b0 2024-09-12T01:34:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.835+0000 7fef53fff640 1 -- 172.21.15.16:0/1742998003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef600096a0 con 0x7fef6c0cb3b0 2024-09-12T01:34:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.835+0000 7fef71b13640 1 -- 172.21.15.16:0/1742998003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c0c4450 con 0x7fef6c0cb3b0 2024-09-12T01:34:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.835+0000 7fef53fff640 1 -- 172.21.15.16:0/1742998003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60009a70 con 0x7fef6c0cb3b0 2024-09-12T01:34:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.835+0000 7fef71b13640 1 -- 172.21.15.16:0/1742998003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c0c4a10 con 0x7fef6c0cb3b0 2024-09-12T01:34:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.836+0000 7fef71b13640 1 -- 172.21.15.16:0/1742998003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6c0ce860 con 0x7fef6c0cb3b0 2024-09-12T01:34:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.837+0000 7fef53fff640 1 -- 172.21.15.16:0/1742998003 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef6001b020 con 0x7fef6c0cb3b0 2024-09-12T01:34:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.837+0000 7fef53fff640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fef5806be30 0x7fef5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.837+0000 7fef53fff640 1 -- 172.21.15.16:0/1742998003 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fef6008b360 con 0x7fef6c0cb3b0 2024-09-12T01:34:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.837+0000 7fef6b7fe640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fef5806be30 0x7fef5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.838+0000 7fef6b7fe640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fef5806be30 0x7fef5806e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fef5c034000 tx=0x7fef5c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:18.842+0000 7fef53fff640 1 -- 172.21.15.16:0/1742998003 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fef60022030 con 0x7fef6c0cb3b0 2024-09-12T01:34:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.053+0000 7fef71b13640 1 -- 172.21.15.16:0/1742998003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef6c06e020 con 0x7fef6c0cb3b0 2024-09-12T01:34:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.054+0000 7fef53fff640 1 -- 172.21.15.16:0/1742998003 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5214 (secure 0 0 0) 0x7fef60009e20 con 0x7fef6c0cb3b0 2024-09-12T01:34:19.054 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:19.054 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-12T01:34:10:019140+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/842803487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":842803487},{"type":"v1","addr":"172.21.15.33:6833","nonce":842803487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/70052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":70052721},{"type":"v1","addr":"172.21.15.33:6835","nonce":70052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2095784417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2095784417},{"type":"v1","addr":"172.21.15.16:6835","nonce":2095784417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:34:06.918906+0000","modified":"2024-09-12T01:34:09.376867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4932":{"gid":4932,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/4214019228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4214019228},{"type":"v1","addr":"172.21.15.16:6837","nonce":4214019228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4932,"qdb_cluster":[4932]},"id":2}]} 2024-09-12T01:34:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-12T01:34:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.056+0000 7fef71b13640 1 -- 172.21.15.16:0/1742998003 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fef5806be30 msgr2=0x7fef5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.056+0000 7fef71b13640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fef5806be30 0x7fef5806e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fef5c034000 tx=0x7fef5c02e000 comp rx=0 tx=0).stop 2024-09-12T01:34:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.056+0000 7fef71b13640 1 -- 172.21.15.16:0/1742998003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c0cb3b0 msgr2=0x7fef6c0c3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.056+0000 7fef71b13640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c0cb3b0 0x7fef6c0c3c70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fef6000d830 tx=0x7fef6000dcf0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.056+0000 7fef71b13640 1 -- 172.21.15.16:0/1742998003 shutdown_connections 2024-09-12T01:34:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.056+0000 7fef71b13640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fef5806be30 0x7fef5806e2f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.056+0000 7fef71b13640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c15e8d0 0x7fef6c0b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.056+0000 7fef71b13640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c0cb3b0 0x7fef6c0c3c70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.056+0000 7fef71b13640 1 --2- 172.21.15.16:0/1742998003 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fef6c0acdd0 0x7fef6c15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.056+0000 7fef71b13640 1 -- 172.21.15.16:0/1742998003 >> 172.21.15.16:0/1742998003 conn(0x7fef6c0cffa0 msgr2=0x7fef6c15d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.056+0000 7fef71b13640 1 -- 172.21.15.16:0/1742998003 shutdown_connections 2024-09-12T01:34:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.056+0000 7fef71b13640 1 -- 172.21.15.16:0/1742998003 wait complete. 2024-09-12T01:34:19.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-12T01:34:19.198 INFO:teuthology.orchestra.run.smithi016.stdout:{"message":"","return_code":0} 2024-09-12T01:34:19.198 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-12T01:34:19.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:34:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.301+0000 7f0704e5c640 1 Processor -- start 2024-09-12T01:34:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.301+0000 7f0704e5c640 1 -- start start 2024-09-12T01:34:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.302+0000 7f0704e5c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f070015e560 0x7f07000ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.302+0000 7f0704e5c640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f07000d1550 0x7f07000b7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.302+0000 7f0704e5c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07000d3480 con 0x7f07000cf510 2024-09-12T01:34:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.302+0000 7f0704e5c640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f07000d3760 con 0x7f070015e560 2024-09-12T01:34:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.302+0000 7f0704e5c640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f07000d3a40 con 0x7f07000d1550 2024-09-12T01:34:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.302+0000 7f06fdd9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f07000cf510 0x7f07000d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49256/0 (socket says 172.21.15.16:49256) 2024-09-12T01:34:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.302+0000 7f06fdd9b640 1 -- 172.21.15.16:0/3067382536 learned_addr learned my addr 172.21.15.16:0/3067382536 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.302+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1889865283 0 0) 0x7f07000d3480 con 0x7f07000cf510 2024-09-12T01:34:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.302+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06ec003520 con 0x7f07000cf510 2024-09-12T01:34:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.302+0000 7f06fe59c640 1 --2- 172.21.15.16:0/3067382536 >> v2:172.21.15.33:3300/0 conn(0x7f070015e560 0x7f07000ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.302+0000 7f06fed9d640 1 --2- 172.21.15.16:0/3067382536 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f07000d1550 0x7f07000b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3555366349 0 0) 0x7f06ec003520 con 0x7f07000cf510 2024-09-12T01:34:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f07000d1550 msgr2=0x7f07000b7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 --2- 172.21.15.16:0/3067382536 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f07000d1550 0x7f07000b7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 >> v2:172.21.15.33:3300/0 conn(0x7f070015e560 msgr2=0x7f07000ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 --2- 172.21.15.16:0/3067382536 >> v2:172.21.15.33:3300/0 conn(0x7f070015e560 0x7f07000ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07000d3d20 con 0x7f07000cf510 2024-09-12T01:34:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f06f0001670 con 0x7f07000cf510 2024-09-12T01:34:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 >> v1:172.21.15.16:6789/0 conn(0x7f07000cf510 legacy=0x7f07000d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 --2- 172.21.15.16:0/3067382536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06ec003a20 0x7f06ec003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 --2- 172.21.15.16:0/3067382536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06ec004800 0x7f06ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 --2- 172.21.15.16:0/3067382536 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f07000d1550 0x7f06ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07000d3760 con 0x7f06ec003a20 2024-09-12T01:34:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07000d3a40 con 0x7f06ec004800 2024-09-12T01:34:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f06ec00b9c0 con 0x7f07000d1550 2024-09-12T01:34:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fe59c640 1 --2- 172.21.15.16:0/3067382536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06ec003a20 0x7f06ec003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fdd9b640 1 --2- 172.21.15.16:0/3067382536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06ec004800 0x7f06ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fed9d640 1 --2- 172.21.15.16:0/3067382536 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f07000d1550 0x7f06ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fe59c640 1 -- 172.21.15.16:0/3067382536 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f07000d1550 msgr2=0x7f06ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fe59c640 1 --2- 172.21.15.16:0/3067382536 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f07000d1550 0x7f06ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fe59c640 1 -- 172.21.15.16:0/3067382536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06ec004800 msgr2=0x7f06ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fe59c640 1 --2- 172.21.15.16:0/3067382536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06ec004800 0x7f06ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.303+0000 7f06fe59c640 1 -- 172.21.15.16:0/3067382536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ec003520 con 0x7f06ec003a20 2024-09-12T01:34:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.304+0000 7f06fe59c640 1 --2- 172.21.15.16:0/3067382536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06ec003a20 0x7f06ec003e40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009af0 tx=0x7f06ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211bc62a486f6b66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.304+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8036070 con 0x7f06ec003a20 2024-09-12T01:34:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.304+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e8031400 con 0x7f06ec003a20 2024-09-12T01:34:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.304+0000 7f06fd59a640 1 -- 172.21.15.16:0/3067382536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8009690 con 0x7f06ec003a20 2024-09-12T01:34:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.305+0000 7f0704e5c640 1 -- 172.21.15.16:0/3067382536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06ec003a20 msgr2=0x7f06ec003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.305+0000 7f0704e5c640 1 --2- 172.21.15.16:0/3067382536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06ec003a20 0x7f06ec003e40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009af0 tx=0x7f06ec005830 comp rx=0 tx=0).stop 2024-09-12T01:34:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.305+0000 7f06fed9d640 1 -- 172.21.15.16:0/3067382536 reap_dead start 2024-09-12T01:34:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.305+0000 7f0704e5c640 1 -- 172.21.15.16:0/3067382536 shutdown_connections 2024-09-12T01:34:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.305+0000 7f0704e5c640 1 -- 172.21.15.16:0/3067382536 >> 172.21.15.16:0/3067382536 conn(0x7f0700073850 msgr2=0x7f070006cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.305+0000 7f0704e5c640 1 -- 172.21.15.16:0/3067382536 shutdown_connections 2024-09-12T01:34:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.305+0000 7f0704e5c640 1 -- 172.21.15.16:0/3067382536 wait complete. 2024-09-12T01:34:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.306+0000 7f0704e5c640 1 Processor -- start 2024-09-12T01:34:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.306+0000 7f0704e5c640 1 -- start start 2024-09-12T01:34:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f0704e5c640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f06ec003a20 0x7f07000afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f0704e5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700160e00 0x7f0700161280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f0704e5c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f070015a0c0 0x7f070015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f0704e5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f070015eac0 con 0x7f0700160e00 2024-09-12T01:34:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f0704e5c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f070015e940 con 0x7f070015a0c0 2024-09-12T01:34:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f0704e5c640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f07000cf980 con 0x7f06ec003a20 2024-09-12T01:34:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f06fdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700160e00 0x7f0700161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f06fdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700160e00 0x7f0700161280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46082/0 (socket says 172.21.15.16:46082) 2024-09-12T01:34:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f06fdd9b640 1 -- 172.21.15.16:0/3705524333 learned_addr learned my addr 172.21.15.16:0/3705524333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f06fe59c640 1 --2- 172.21.15.16:0/3705524333 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f06ec003a20 0x7f07000afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f06fdd9b640 1 -- 172.21.15.16:0/3705524333 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f06ec003a20 msgr2=0x7f07000afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f06fdd9b640 1 --2- 172.21.15.16:0/3705524333 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f06ec003a20 0x7f07000afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f06fdd9b640 1 -- 172.21.15.16:0/3705524333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f070015a0c0 msgr2=0x7f070015a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f06fdd9b640 1 --2- 172.21.15.16:0/3705524333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f070015a0c0 0x7f070015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.307+0000 7f06fdd9b640 1 -- 172.21.15.16:0/3705524333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07000abc50 con 0x7f0700160e00 2024-09-12T01:34:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.308+0000 7f06fdd9b640 1 --2- 172.21.15.16:0/3705524333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700160e00 0x7f0700161280 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f06f0007310 tx=0x7f06f00077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.308+0000 7f06f6ffd640 1 -- 172.21.15.16:0/3705524333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f000f940 con 0x7f0700160e00 2024-09-12T01:34:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.308+0000 7f06f6ffd640 1 -- 172.21.15.16:0/3705524333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f0018070 con 0x7f0700160e00 2024-09-12T01:34:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.308+0000 7f06f6ffd640 1 -- 172.21.15.16:0/3705524333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f000fd00 con 0x7f0700160e00 2024-09-12T01:34:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.308+0000 7f0704e5c640 1 -- 172.21.15.16:0/3705524333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07000aec60 con 0x7f0700160e00 2024-09-12T01:34:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.308+0000 7f0704e5c640 1 -- 172.21.15.16:0/3705524333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07000af220 con 0x7f0700160e00 2024-09-12T01:34:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.309+0000 7f0704e5c640 1 -- 172.21.15.16:0/3705524333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07000ceee0 con 0x7f0700160e00 2024-09-12T01:34:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.309+0000 7f06f6ffd640 1 -- 172.21.15.16:0/3705524333 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f06f001c410 con 0x7f0700160e00 2024-09-12T01:34:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.310+0000 7f06f6ffd640 1 --2- 172.21.15.16:0/3705524333 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f06e006be60 0x7f06e006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.310+0000 7f06f6ffd640 1 -- 172.21.15.16:0/3705524333 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f06f008ade0 con 0x7f0700160e00 2024-09-12T01:34:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.310+0000 7f06fe59c640 1 --2- 172.21.15.16:0/3705524333 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f06e006be60 0x7f06e006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.310+0000 7f06fe59c640 1 --2- 172.21.15.16:0/3705524333 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f06e006be60 0x7f06e006e320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f06e8033000 tx=0x7f06e8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.314+0000 7f06f6ffd640 1 -- 172.21.15.16:0/3705524333 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f06f0059560 con 0x7f0700160e00 2024-09-12T01:34:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.525+0000 7f0704e5c640 1 -- 172.21.15.16:0/3705524333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f070015d190 con 0x7f0700160e00 2024-09-12T01:34:19.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.526+0000 7f06f6ffd640 1 -- 172.21.15.16:0/3705524333 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5214 (secure 0 0 0) 0x7f06f0058e60 con 0x7f0700160e00 2024-09-12T01:34:19.526 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:19.526 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-12T01:34:10:019140+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/842803487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":842803487},{"type":"v1","addr":"172.21.15.33:6833","nonce":842803487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/70052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":70052721},{"type":"v1","addr":"172.21.15.33:6835","nonce":70052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2095784417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2095784417},{"type":"v1","addr":"172.21.15.16:6835","nonce":2095784417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:34:06.918906+0000","modified":"2024-09-12T01:34:09.376867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4932":{"gid":4932,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/4214019228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4214019228},{"type":"v1","addr":"172.21.15.16:6837","nonce":4214019228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4932,"qdb_cluster":[4932]},"id":2}]} 2024-09-12T01:34:19.526 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-12T01:34:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.528+0000 7f0704e5c640 1 -- 172.21.15.16:0/3705524333 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f06e006be60 msgr2=0x7f06e006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.528+0000 7f0704e5c640 1 --2- 172.21.15.16:0/3705524333 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f06e006be60 0x7f06e006e320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f06e8033000 tx=0x7f06e8008030 comp rx=0 tx=0).stop 2024-09-12T01:34:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.528+0000 7f0704e5c640 1 -- 172.21.15.16:0/3705524333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700160e00 msgr2=0x7f0700161280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.528+0000 7f0704e5c640 1 --2- 172.21.15.16:0/3705524333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700160e00 0x7f0700161280 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f06f0007310 tx=0x7f06f00077d0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.528+0000 7f0704e5c640 1 -- 172.21.15.16:0/3705524333 shutdown_connections 2024-09-12T01:34:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.528+0000 7f0704e5c640 1 --2- 172.21.15.16:0/3705524333 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f06e006be60 0x7f06e006e320 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.528+0000 7f0704e5c640 1 --2- 172.21.15.16:0/3705524333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f070015a0c0 0x7f070015a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.528+0000 7f0704e5c640 1 --2- 172.21.15.16:0/3705524333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0700160e00 0x7f0700161280 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.528+0000 7f0704e5c640 1 --2- 172.21.15.16:0/3705524333 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f06ec003a20 0x7f07000afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.528+0000 7f0704e5c640 1 -- 172.21.15.16:0/3705524333 >> 172.21.15.16:0/3705524333 conn(0x7f0700073850 msgr2=0x7f07000cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.528+0000 7f0704e5c640 1 -- 172.21.15.16:0/3705524333 shutdown_connections 2024-09-12T01:34:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19.528+0000 7f0704e5c640 1 -- 172.21.15.16:0/3705524333 wait complete. 2024-09-12T01:34:19.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_log_pause 1 2024-09-12T01:34:19.655 INFO:teuthology.orchestra.run.smithi016.stdout:{"success":"mds_log_pause = '' "} 2024-09-12T01:34:19.656 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_log_pause = '' " } 2024-09-12T01:34:19.656 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-12T01:34:19.656 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-12T01:34:19.656 DEBUG:teuthology.orchestra.run.smithi016:> os.chdir("/home/ubuntu/cephtest/mnt.0/testdir") 2024-09-12T01:34:19.656 DEBUG:teuthology.orchestra.run.smithi016:> os.setgid(65534) 2024-09-12T01:34:19.656 DEBUG:teuthology.orchestra.run.smithi016:> os.setuid(65534) 2024-09-12T01:34:19.656 DEBUG:teuthology.orchestra.run.smithi016:> fd = os.open("testfile", os.O_CREAT | os.O_RDWR, 0o644) 2024-09-12T01:34:19.657 DEBUG:teuthology.orchestra.run.smithi016:> os.fchmod(fd, 0o640) 2024-09-12T01:34:19.657 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-12T01:34:19.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:34:19.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:19.745 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod 600 /home/ubuntu/cephtest/mnt.0/testdir/testfile) 2024-09-12T01:34:29.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 2:0 2024-09-12T01:34:29.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.907+0000 7f2412cd2640 1 Processor -- start 2024-09-12T01:34:29.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.907+0000 7f2412cd2640 1 -- start start 2024-09-12T01:34:29.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.908+0000 7f2412cd2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f240c147b70 0x7f240c0b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.908+0000 7f2412cd2640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f240c06c200 0x7f240c161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.908+0000 7f2412cd2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f240c162660 con 0x7f240c15e9f0 2024-09-12T01:34:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.908+0000 7f2412cd2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f240c162430 con 0x7f240c147b70 2024-09-12T01:34:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.908+0000 7f2412cd2640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f240c070330 con 0x7f240c06c200 2024-09-12T01:34:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.908+0000 7f2410a6e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f240c147b70 0x7f240c0b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.908+0000 7f2410a6e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f240c147b70 0x7f240c0b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36332/0 (socket says 172.21.15.16:36332) 2024-09-12T01:34:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.908+0000 7f2410a6e640 1 -- 172.21.15.16:0/823422692 learned_addr learned my addr 172.21.15.16:0/823422692 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.908+0000 7f241126f640 1 --2- 172.21.15.16:0/823422692 >> v2:172.21.15.33:3300/0 conn(0x7f240c06c200 0x7f240c161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.909+0000 7f2410a6e640 1 -- 172.21.15.16:0/823422692 >> v2:172.21.15.33:3300/0 conn(0x7f240c06c200 msgr2=0x7f240c161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.909+0000 7f2410a6e640 1 --2- 172.21.15.16:0/823422692 >> v2:172.21.15.33:3300/0 conn(0x7f240c06c200 0x7f240c161ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.909+0000 7f2410a6e640 1 -- 172.21.15.16:0/823422692 >> v1:172.21.15.16:6789/0 conn(0x7f240c15e9f0 legacy=0x7f240c06baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.909+0000 7f2410a6e640 1 -- 172.21.15.16:0/823422692 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c162940 con 0x7f240c147b70 2024-09-12T01:34:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.909+0000 7f240b7fe640 1 -- 172.21.15.16:0/823422692 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1345461685 0 0) 0x7f240c162660 con 0x7f240c15e9f0 2024-09-12T01:34:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.909+0000 7f2410a6e640 1 --2- 172.21.15.16:0/823422692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c147b70 0x7f240c0b4b20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f240c15ee80 tx=0x7f23fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e146fb65712d1368 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.910+0000 7f240b7fe640 1 -- 172.21.15.16:0/823422692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc037070 con 0x7f240c147b70 2024-09-12T01:34:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.910+0000 7f240b7fe640 1 -- 172.21.15.16:0/823422692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23fc032440 con 0x7f240c147b70 2024-09-12T01:34:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.910+0000 7f240b7fe640 1 -- 172.21.15.16:0/823422692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc032810 con 0x7f240c147b70 2024-09-12T01:34:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.910+0000 7f2412cd2640 1 -- 172.21.15.16:0/823422692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c147b70 msgr2=0x7f240c0b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.910+0000 7f2412cd2640 1 --2- 172.21.15.16:0/823422692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c147b70 0x7f240c0b4b20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f240c15ee80 tx=0x7f23fc02a270 comp rx=0 tx=0).stop 2024-09-12T01:34:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.910+0000 7f2412cd2640 1 -- 172.21.15.16:0/823422692 shutdown_connections 2024-09-12T01:34:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.910+0000 7f2412cd2640 1 --2- 172.21.15.16:0/823422692 >> v2:172.21.15.33:3300/0 conn(0x7f240c06c200 0x7f240c161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.910+0000 7f2412cd2640 1 --2- 172.21.15.16:0/823422692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c147b70 0x7f240c0b4b20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.910+0000 7f2412cd2640 1 -- 172.21.15.16:0/823422692 >> 172.21.15.16:0/823422692 conn(0x7f240c0b2e10 msgr2=0x7f240c0b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.910+0000 7f2412cd2640 1 -- 172.21.15.16:0/823422692 shutdown_connections 2024-09-12T01:34:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.910+0000 7f2412cd2640 1 -- 172.21.15.16:0/823422692 wait complete. 2024-09-12T01:34:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.911+0000 7f2412cd2640 1 Processor -- start 2024-09-12T01:34:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.912+0000 7f2412cd2640 1 -- start start 2024-09-12T01:34:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.912+0000 7f2412cd2640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f240c06c200 0x7f240c1f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.912+0000 7f2412cd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c147b70 0x7f240c15dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.912+0000 7f2412cd2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f240c15e9f0 0x7f240c142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.912+0000 7f2412cd2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f240c0707c0 con 0x7f240c147b70 2024-09-12T01:34:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f2412cd2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f240c147f50 con 0x7f240c15e9f0 2024-09-12T01:34:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f2412cd2640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f240c070940 con 0x7f240c06c200 2024-09-12T01:34:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f240bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c147b70 0x7f240c15dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f240bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c147b70 0x7f240c15dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36336/0 (socket says 172.21.15.16:36336) 2024-09-12T01:34:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f240bfff640 1 -- 172.21.15.16:0/1531875246 learned_addr learned my addr 172.21.15.16:0/1531875246 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f2410a6e640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f240c06c200 0x7f240c1f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f241126f640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f240c15e9f0 0x7f240c142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f240bfff640 1 -- 172.21.15.16:0/1531875246 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f240c06c200 msgr2=0x7f240c1f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f240bfff640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f240c06c200 0x7f240c1f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f240bfff640 1 -- 172.21.15.16:0/1531875246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f240c15e9f0 msgr2=0x7f240c142500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f240bfff640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f240c15e9f0 0x7f240c142500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f240bfff640 1 -- 172.21.15.16:0/1531875246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c0d03d0 con 0x7f240c147b70 2024-09-12T01:34:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f2410a6e640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f240c06c200 0x7f240c1f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f240bfff640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c147b70 0x7f240c15dc60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f240000c410 tx=0x7f240000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f24097fa640 1 -- 172.21.15.16:0/1531875246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400017070 con 0x7f240c147b70 2024-09-12T01:34:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f24097fa640 1 -- 172.21.15.16:0/1531875246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24000035f0 con 0x7f240c147b70 2024-09-12T01:34:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f2412cd2640 1 -- 172.21.15.16:0/1531875246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240c1f7ef0 con 0x7f240c147b70 2024-09-12T01:34:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f24097fa640 1 -- 172.21.15.16:0/1531875246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000039e0 con 0x7f240c147b70 2024-09-12T01:34:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.913+0000 7f2412cd2640 1 -- 172.21.15.16:0/1531875246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f240c1f8470 con 0x7f240c147b70 2024-09-12T01:34:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.914+0000 7f2412cd2640 1 -- 172.21.15.16:0/1531875246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240c143a50 con 0x7f240c147b70 2024-09-12T01:34:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.915+0000 7f24097fa640 1 -- 172.21.15.16:0/1531875246 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f240001a070 con 0x7f240c147b70 2024-09-12T01:34:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.915+0000 7f24097fa640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f23f806be30 0x7f23f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.915+0000 7f24097fa640 1 -- 172.21.15.16:0/1531875246 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f240008a420 con 0x7f240c147b70 2024-09-12T01:34:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.915+0000 7f2410a6e640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f23f806be30 0x7f23f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.916+0000 7f2410a6e640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f23f806be30 0x7f23f806e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23fc034000 tx=0x7f23fc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:29.920+0000 7f24097fa640 1 -- 172.21.15.16:0/1531875246 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2400058c40 con 0x7f240c147b70 2024-09-12T01:34:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.164+0000 7f2412cd2640 1 -- 172.21.15.16:0/1531875246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "2:0"} v 0) -- 0x7f240c161590 con 0x7f240c147b70 2024-09-12T01:34:30.164 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-12T01:34:30.164+0000 7fe0e42bd640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T01:34:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.166+0000 7f24097fa640 1 -- 172.21.15.16:0/1531875246 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "2:0"}]=0 failed mds gid 4932 v17) ==== 97+0+0 (secure 0 0 0) 0x7f2400058540 con 0x7f240c147b70 2024-09-12T01:34:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:failed mds gid 4932 2024-09-12T01:34:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.169+0000 7f2412cd2640 1 -- 172.21.15.16:0/1531875246 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f23f806be30 msgr2=0x7f23f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.169+0000 7f2412cd2640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f23f806be30 0x7f23f806e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23fc034000 tx=0x7f23fc02e000 comp rx=0 tx=0).stop 2024-09-12T01:34:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.169+0000 7f2412cd2640 1 -- 172.21.15.16:0/1531875246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c147b70 msgr2=0x7f240c15dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.169+0000 7f2412cd2640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c147b70 0x7f240c15dc60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f240000c410 tx=0x7f240000c8d0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.169+0000 7f2412cd2640 1 -- 172.21.15.16:0/1531875246 shutdown_connections 2024-09-12T01:34:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.169+0000 7f2412cd2640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f23f806be30 0x7f23f806e2f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.169+0000 7f2412cd2640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f240c15e9f0 0x7f240c142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.169+0000 7f2412cd2640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c147b70 0x7f240c15dc60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.169+0000 7f2412cd2640 1 --2- 172.21.15.16:0/1531875246 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f240c06c200 0x7f240c1f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.169+0000 7f2412cd2640 1 -- 172.21.15.16:0/1531875246 >> 172.21.15.16:0/1531875246 conn(0x7f240c0b2e10 msgr2=0x7f240c0b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.169+0000 7f2412cd2640 1 -- 172.21.15.16:0/1531875246 shutdown_connections 2024-09-12T01:34:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.169+0000 7f2412cd2640 1 -- 172.21.15.16:0/1531875246 wait complete. 2024-09-12T01:34:30.170 INFO:tasks.ceph.mds.c.smithi016.stderr: -1967> 2024-09-12T01:34:12.542+0000 7f897292b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T01:34:30.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:34:30.203 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-12T01:34:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.289+0000 7f33bf82e640 1 Processor -- start 2024-09-12T01:34:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.290+0000 7f33bf82e640 1 -- start start 2024-09-12T01:34:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.290+0000 7f33bf82e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33b815e510 0x7f33b80ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.290+0000 7f33bf82e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f33b80cf040 0x7f33b80d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.290+0000 7f33bf82e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f33b80d1200 0x7f33b80b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.290+0000 7f33bf82e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f33b80d3430 con 0x7f33b815e510 2024-09-12T01:34:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.290+0000 7f33bf82e640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f33b80d1690 con 0x7f33b80cf040 2024-09-12T01:34:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.290+0000 7f33bf82e640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f33b80d36d0 con 0x7f33b80d1200 2024-09-12T01:34:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.291+0000 7f33bd5ca640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33b815e510 0x7f33b80ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.291+0000 7f33bd5ca640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33b815e510 0x7f33b80ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36360/0 (socket says 172.21.15.16:36360) 2024-09-12T01:34:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.291+0000 7f33bd5ca640 1 -- 172.21.15.16:0/713354965 learned_addr learned my addr 172.21.15.16:0/713354965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.291+0000 7f33bddcb640 1 --2- 172.21.15.16:0/713354965 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f33b80d1200 0x7f33b80b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.291+0000 7f33bcdc9640 1 --2- 172.21.15.16:0/713354965 >> v2:172.21.15.33:3300/0 conn(0x7f33b80cf040 0x7f33b80d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:30.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.291+0000 7f33bd5ca640 1 -- 172.21.15.16:0/713354965 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f33b80d1200 msgr2=0x7f33b80b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:30.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.291+0000 7f33bd5ca640 1 --2- 172.21.15.16:0/713354965 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f33b80d1200 0x7f33b80b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.291+0000 7f33bd5ca640 1 -- 172.21.15.16:0/713354965 >> v2:172.21.15.33:3300/0 conn(0x7f33b80cf040 msgr2=0x7f33b80d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:30.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.291+0000 7f33bd5ca640 1 --2- 172.21.15.16:0/713354965 >> v2:172.21.15.33:3300/0 conn(0x7f33b80cf040 0x7f33b80d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.291+0000 7f33bd5ca640 1 -- 172.21.15.16:0/713354965 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b80d39b0 con 0x7f33b815e510 2024-09-12T01:34:30.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.291+0000 7f33bddcb640 1 --2- 172.21.15.16:0/713354965 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f33b80d1200 0x7f33b80b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.292+0000 7f33bd5ca640 1 --2- 172.21.15.16:0/713354965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b815e510 0x7f33b80ceb00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f33b80d4460 tx=0x7f33b402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d364b90ba5f77b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.292+0000 7f33affff640 1 -- 172.21.15.16:0/713354965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4037070 con 0x7f33b815e510 2024-09-12T01:34:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.292+0000 7f33affff640 1 -- 172.21.15.16:0/713354965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b4032440 con 0x7f33b815e510 2024-09-12T01:34:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.292+0000 7f33affff640 1 -- 172.21.15.16:0/713354965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4032810 con 0x7f33b815e510 2024-09-12T01:34:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.292+0000 7f33bf82e640 1 -- 172.21.15.16:0/713354965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b815e510 msgr2=0x7f33b80ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.292+0000 7f33bf82e640 1 --2- 172.21.15.16:0/713354965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b815e510 0x7f33b80ceb00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f33b80d4460 tx=0x7f33b402a140 comp rx=0 tx=0).stop 2024-09-12T01:34:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.292+0000 7f33bf82e640 1 -- 172.21.15.16:0/713354965 shutdown_connections 2024-09-12T01:34:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.292+0000 7f33bf82e640 1 --2- 172.21.15.16:0/713354965 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f33b80d1200 0x7f33b80b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.292+0000 7f33bf82e640 1 --2- 172.21.15.16:0/713354965 >> v2:172.21.15.33:3300/0 conn(0x7f33b80cf040 0x7f33b80d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.292+0000 7f33bf82e640 1 --2- 172.21.15.16:0/713354965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b815e510 0x7f33b80ceb00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.292+0000 7f33bf82e640 1 -- 172.21.15.16:0/713354965 >> 172.21.15.16:0/713354965 conn(0x7f33b806d1a0 msgr2=0x7f33b8073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.292+0000 7f33bf82e640 1 -- 172.21.15.16:0/713354965 shutdown_connections 2024-09-12T01:34:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.293+0000 7f33bf82e640 1 -- 172.21.15.16:0/713354965 wait complete. 2024-09-12T01:34:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.294+0000 7f33bf82e640 1 Processor -- start 2024-09-12T01:34:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.294+0000 7f33bf82e640 1 -- start start 2024-09-12T01:34:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.294+0000 7f33bf82e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b80cf040 0x7f33b80ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.294+0000 7f33bf82e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f33b80d1200 0x7f33b81f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.294+0000 7f33bf82e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33b815e510 0x7f33b81f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.294+0000 7f33bf82e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33b80d22a0 con 0x7f33b80cf040 2024-09-12T01:34:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.294+0000 7f33bf82e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33b80d3f70 con 0x7f33b815e510 2024-09-12T01:34:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.294+0000 7f33bf82e640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f33b815e8f0 con 0x7f33b80d1200 2024-09-12T01:34:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33bd5ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b80cf040 0x7f33b80ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33bd5ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b80cf040 0x7f33b80ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36370/0 (socket says 172.21.15.16:36370) 2024-09-12T01:34:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33bd5ca640 1 -- 172.21.15.16:0/3033634898 learned_addr learned my addr 172.21.15.16:0/3033634898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33bddcb640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33b815e510 0x7f33b81f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33bcdc9640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f33b80d1200 0x7f33b81f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33bd5ca640 1 -- 172.21.15.16:0/3033634898 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f33b80d1200 msgr2=0x7f33b81f79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33bd5ca640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f33b80d1200 0x7f33b81f79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33bd5ca640 1 -- 172.21.15.16:0/3033634898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33b815e510 msgr2=0x7f33b81f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33bd5ca640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33b815e510 0x7f33b81f7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33bd5ca640 1 -- 172.21.15.16:0/3033634898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b8160760 con 0x7f33b80cf040 2024-09-12T01:34:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33bd5ca640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b80cf040 0x7f33b80ca890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f33b402a180 tx=0x7f33b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33adffb640 1 -- 172.21.15.16:0/3033634898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4041020 con 0x7f33b80cf040 2024-09-12T01:34:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33adffb640 1 -- 172.21.15.16:0/3033634898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b40306c0 con 0x7f33b80cf040 2024-09-12T01:34:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33bf82e640 1 -- 172.21.15.16:0/3033634898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b8160a40 con 0x7f33b80cf040 2024-09-12T01:34:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33adffb640 1 -- 172.21.15.16:0/3033634898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4037040 con 0x7f33b80cf040 2024-09-12T01:34:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.295+0000 7f33bf82e640 1 -- 172.21.15.16:0/3033634898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b8161000 con 0x7f33b80cf040 2024-09-12T01:34:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.297+0000 7f33bf82e640 1 -- 172.21.15.16:0/3033634898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b80d0cc0 con 0x7f33b80cf040 2024-09-12T01:34:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.297+0000 7f33adffb640 1 -- 172.21.15.16:0/3033634898 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33b4044020 con 0x7f33b80cf040 2024-09-12T01:34:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.297+0000 7f33adffb640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f339c06be30 0x7f339c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.298+0000 7f33adffb640 1 -- 172.21.15.16:0/3033634898 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f33b40ab7b0 con 0x7f33b80cf040 2024-09-12T01:34:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.298+0000 7f33bcdc9640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f339c06be30 0x7f339c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.298+0000 7f33bcdc9640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f339c06be30 0x7f339c06e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f33b81518e0 tx=0x7f33a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.302+0000 7f33adffb640 1 -- 172.21.15.16:0/3033634898 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f33b404b030 con 0x7f33b80cf040 2024-09-12T01:34:30.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.512+0000 7f33bf82e640 1 -- 172.21.15.16:0/3033634898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33b80cadd0 con 0x7f33b80cf040 2024-09-12T01:34:30.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.513+0000 7f33adffb640 1 -- 172.21.15.16:0/3033634898 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+4383 (secure 0 0 0) 0x7f33b4079930 con 0x7f33b80cf040 2024-09-12T01:34:30.513 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:30.513 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"btime":"2024-09-12T01:34:30:169887+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/842803487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":842803487},{"type":"v1","addr":"172.21.15.33:6833","nonce":842803487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/70052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":70052721},{"type":"v1","addr":"172.21.15.33:6835","nonce":70052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:34:06.918906+0000","modified":"2024-09-12T01:34:30.169871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"a","rank":0,"incarnation":18,"state":"up:replay","state_seq":1,"addr":"172.21.15.16:6835/2095784417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2095784417},{"type":"v1","addr":"172.21.15.16:6835","nonce":2095784417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-12T01:34:30.513 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 18 2024-09-12T01:34:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.515+0000 7f33bf82e640 1 -- 172.21.15.16:0/3033634898 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f339c06be30 msgr2=0x7f339c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.515+0000 7f33bf82e640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f339c06be30 0x7f339c06e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f33b81518e0 tx=0x7f33a8008040 comp rx=0 tx=0).stop 2024-09-12T01:34:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.515+0000 7f33bf82e640 1 -- 172.21.15.16:0/3033634898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b80cf040 msgr2=0x7f33b80ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.515+0000 7f33bf82e640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b80cf040 0x7f33b80ca890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f33b402a180 tx=0x7f33b40046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.515+0000 7f33bf82e640 1 -- 172.21.15.16:0/3033634898 shutdown_connections 2024-09-12T01:34:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.515+0000 7f33bf82e640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f339c06be30 0x7f339c06e2f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.515+0000 7f33bf82e640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33b815e510 0x7f33b81f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.515+0000 7f33bf82e640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f33b80d1200 0x7f33b81f79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.515+0000 7f33bf82e640 1 --2- 172.21.15.16:0/3033634898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b80cf040 0x7f33b80ca890 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.515+0000 7f33bf82e640 1 -- 172.21.15.16:0/3033634898 >> 172.21.15.16:0/3033634898 conn(0x7f33b806d1a0 msgr2=0x7f33b80b7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:30.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.515+0000 7f33bf82e640 1 -- 172.21.15.16:0/3033634898 shutdown_connections 2024-09-12T01:34:30.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:30.515+0000 7f33bf82e640 1 -- 172.21.15.16:0/3033634898 wait complete. 2024-09-12T01:34:30.928 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T01:34:06.918906+0000', 'modified': '2024-09-12T01:34:30.169871+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4929}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4929': {'gid': 4929, 'name': 'a', 'rank': 0, 'incarnation': 18, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.16:6835/2095784417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 2095784417}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2095784417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-12T01:34:30.929 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4929:up:replay 2024-09-12T01:34:31.930 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:34:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.042+0000 7ff829591640 1 Processor -- start 2024-09-12T01:34:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.042+0000 7ff829591640 1 -- start start 2024-09-12T01:34:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.043+0000 7ff829591640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff82415e2e0 0x7ff82415e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.043+0000 7ff829591640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8240d0e10 0x7ff8240d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.043+0000 7ff829591640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8240d25a0 con 0x7ff824162130 2024-09-12T01:34:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.043+0000 7ff829591640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7ff8240d17e0 con 0x7ff8240d0e10 2024-09-12T01:34:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.043+0000 7ff829591640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff8240cf270 con 0x7ff82415e2e0 2024-09-12T01:34:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.044+0000 7ff822ffd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff82415e2e0 0x7ff82415e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.044+0000 7ff8237fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8240d0e10 0x7ff8240d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.044+0000 7ff8227fc640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff824162130 0x7ff8240ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.16:58050/0 (socket says 172.21.15.16:58050) 2024-09-12T01:34:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.044+0000 7ff8227fc640 1 -- 172.21.15.16:0/1306596473 learned_addr learned my addr 172.21.15.16:0/1306596473 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.044+0000 7ff821ffb640 1 -- 172.21.15.16:0/1306596473 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 970616081 0 0) 0x7ff8240d25a0 con 0x7ff824162130 2024-09-12T01:34:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.044+0000 7ff821ffb640 1 -- 172.21.15.16:0/1306596473 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff810003520 con 0x7ff824162130 2024-09-12T01:34:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.044+0000 7ff8237fe640 1 -- 172.21.15.16:0/1306596473 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff82415e2e0 msgr2=0x7ff82415e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.044+0000 7ff8237fe640 1 --2- 172.21.15.16:0/1306596473 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff82415e2e0 0x7ff82415e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.044+0000 7ff8237fe640 1 -- 172.21.15.16:0/1306596473 >> v1:172.21.15.33:6789/0 conn(0x7ff824162130 legacy=0x7ff8240ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.044+0000 7ff8237fe640 1 -- 172.21.15.16:0/1306596473 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8240cf550 con 0x7ff8240d0e10 2024-09-12T01:34:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.044+0000 7ff822ffd640 1 --2- 172.21.15.16:0/1306596473 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff82415e2e0 0x7ff82415e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.045+0000 7ff8237fe640 1 --2- 172.21.15.16:0/1306596473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8240d0e10 0x7ff8240d12a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0097a0 tx=0x7ff81c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7cc68111a598139 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.045+0000 7ff821ffb640 1 -- 172.21.15.16:0/1306596473 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c037070 con 0x7ff8240d0e10 2024-09-12T01:34:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.045+0000 7ff821ffb640 1 -- 172.21.15.16:0/1306596473 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81c02ae50 con 0x7ff8240d0e10 2024-09-12T01:34:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.045+0000 7ff821ffb640 1 -- 172.21.15.16:0/1306596473 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c032640 con 0x7ff8240d0e10 2024-09-12T01:34:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.046+0000 7ff829591640 1 -- 172.21.15.16:0/1306596473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8240d0e10 msgr2=0x7ff8240d12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.046+0000 7ff829591640 1 --2- 172.21.15.16:0/1306596473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8240d0e10 0x7ff8240d12a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0097a0 tx=0x7ff81c02a100 comp rx=0 tx=0).stop 2024-09-12T01:34:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.046+0000 7ff829591640 1 -- 172.21.15.16:0/1306596473 shutdown_connections 2024-09-12T01:34:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.046+0000 7ff829591640 1 --2- 172.21.15.16:0/1306596473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8240d0e10 0x7ff8240d12a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.046+0000 7ff829591640 1 --2- 172.21.15.16:0/1306596473 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff82415e2e0 0x7ff82415e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.046+0000 7ff829591640 1 -- 172.21.15.16:0/1306596473 >> 172.21.15.16:0/1306596473 conn(0x7ff82406cad0 msgr2=0x7ff82406cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.046+0000 7ff829591640 1 -- 172.21.15.16:0/1306596473 shutdown_connections 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.046+0000 7ff829591640 1 -- 172.21.15.16:0/1306596473 wait complete. 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.047+0000 7ff829591640 1 Processor -- start 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.047+0000 7ff829591640 1 -- start start 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff829591640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8240d0e10 0x7ff824159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff829591640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff82415e2e0 0x7ff8240b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff829591640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824162130 0x7ff8240b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff829591640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8240d2aa0 con 0x7ff8240d0e10 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff829591640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8240cfb10 con 0x7ff824162130 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff829591640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff8240d2c20 con 0x7ff82415e2e0 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff822ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8240d0e10 0x7ff824159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff822ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8240d0e10 0x7ff824159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36382/0 (socket says 172.21.15.16:36382) 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff822ffd640 1 -- 172.21.15.16:0/294612953 learned_addr learned my addr 172.21.15.16:0/294612953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff8237fe640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824162130 0x7ff8240b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff8227fc640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff82415e2e0 0x7ff8240b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff822ffd640 1 -- 172.21.15.16:0/294612953 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff82415e2e0 msgr2=0x7ff8240b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff822ffd640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff82415e2e0 0x7ff8240b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff822ffd640 1 -- 172.21.15.16:0/294612953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824162130 msgr2=0x7ff8240b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff822ffd640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824162130 0x7ff8240b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.048+0000 7ff822ffd640 1 -- 172.21.15.16:0/294612953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff824160a90 con 0x7ff8240d0e10 2024-09-12T01:34:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.049+0000 7ff822ffd640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8240d0e10 0x7ff824159500 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff814009f60 tx=0x7ff814009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.049+0000 7ff81b7fe640 1 -- 172.21.15.16:0/294612953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814018020 con 0x7ff8240d0e10 2024-09-12T01:34:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.049+0000 7ff81b7fe640 1 -- 172.21.15.16:0/294612953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff814014680 con 0x7ff8240d0e10 2024-09-12T01:34:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.049+0000 7ff829591640 1 -- 172.21.15.16:0/294612953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8240cccb0 con 0x7ff8240d0e10 2024-09-12T01:34:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.049+0000 7ff81b7fe640 1 -- 172.21.15.16:0/294612953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814014a50 con 0x7ff8240d0e10 2024-09-12T01:34:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.049+0000 7ff829591640 1 -- 172.21.15.16:0/294612953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8240cd270 con 0x7ff8240d0e10 2024-09-12T01:34:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.050+0000 7ff829591640 1 -- 172.21.15.16:0/294612953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8240ba3e0 con 0x7ff8240d0e10 2024-09-12T01:34:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.050+0000 7ff81b7fe640 1 -- 172.21.15.16:0/294612953 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff81401b020 con 0x7ff8240d0e10 2024-09-12T01:34:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.051+0000 7ff81b7fe640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff80406be30 0x7ff80406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.051+0000 7ff81b7fe640 1 -- 172.21.15.16:0/294612953 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7ff81408bec0 con 0x7ff8240d0e10 2024-09-12T01:34:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.051+0000 7ff8227fc640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff80406be30 0x7ff80406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.051+0000 7ff8227fc640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff80406be30 0x7ff80406e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff8240d4ff0 tx=0x7ff80c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.055+0000 7ff81b7fe640 1 -- 172.21.15.16:0/294612953 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff814022020 con 0x7ff8240d0e10 2024-09-12T01:34:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.271+0000 7ff829591640 1 -- 172.21.15.16:0/294612953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8240d4bb0 con 0x7ff8240d0e10 2024-09-12T01:34:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.271+0000 7ff81b7fe640 1 -- 172.21.15.16:0/294612953 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5209 (secure 0 0 0) 0x7ff814010040 con 0x7ff8240d0e10 2024-09-12T01:34:32.271 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:32.272 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-12T01:34:32:183805+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/842803487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":842803487},{"type":"v1","addr":"172.21.15.33:6833","nonce":842803487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/70052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":70052721},{"type":"v1","addr":"172.21.15.33:6835","nonce":70052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3777111462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3777111462},{"type":"v1","addr":"172.21.15.16:6837","nonce":3777111462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:34:06.918906+0000","modified":"2024-09-12T01:34:31.186160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"a","rank":0,"incarnation":18,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.16:6835/2095784417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2095784417},{"type":"v1","addr":"172.21.15.16:6835","nonce":2095784417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-12T01:34:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-12T01:34:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.273+0000 7ff829591640 1 -- 172.21.15.16:0/294612953 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff80406be30 msgr2=0x7ff80406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.273+0000 7ff829591640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff80406be30 0x7ff80406e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff8240d4ff0 tx=0x7ff80c009040 comp rx=0 tx=0).stop 2024-09-12T01:34:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.273+0000 7ff829591640 1 -- 172.21.15.16:0/294612953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8240d0e10 msgr2=0x7ff824159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.273+0000 7ff829591640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8240d0e10 0x7ff824159500 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff814009f60 tx=0x7ff814009f90 comp rx=0 tx=0).stop 2024-09-12T01:34:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.273+0000 7ff829591640 1 -- 172.21.15.16:0/294612953 shutdown_connections 2024-09-12T01:34:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.273+0000 7ff829591640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff80406be30 0x7ff80406e2f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.273+0000 7ff829591640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824162130 0x7ff8240b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.273+0000 7ff829591640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff82415e2e0 0x7ff8240b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.273+0000 7ff829591640 1 --2- 172.21.15.16:0/294612953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8240d0e10 0x7ff824159500 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.273+0000 7ff829591640 1 -- 172.21.15.16:0/294612953 >> 172.21.15.16:0/294612953 conn(0x7ff82406cad0 msgr2=0x7ff8240b9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.273+0000 7ff829591640 1 -- 172.21.15.16:0/294612953 shutdown_connections 2024-09-12T01:34:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:32.273+0000 7ff829591640 1 -- 172.21.15.16:0/294612953 wait complete. 2024-09-12T01:34:32.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T01:34:06.918906+0000', 'modified': '2024-09-12T01:34:31.186160+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4929}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4929': {'gid': 4929, 'name': 'a', 'rank': 0, 'incarnation': 18, 'state': 'up:rejoin', 'state_seq': 10, 'addr': '172.21.15.16:6835/2095784417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 2095784417}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2095784417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-12T01:34:32.688 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4929:up:rejoin 2024-09-12T01:34:33.688 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:34:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.805+0000 7fb051bcf640 1 Processor -- start 2024-09-12T01:34:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.805+0000 7fb051bcf640 1 -- start start 2024-09-12T01:34:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.805+0000 7fb051bcf640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb04c1673d0 0x7fb04c1697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.805+0000 7fb051bcf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb04c073ec0 con 0x7fb04c15e760 2024-09-12T01:34:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.805+0000 7fb051bcf640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb04c072e20 con 0x7fb04c06baf0 2024-09-12T01:34:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.805+0000 7fb051bcf640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb04c0cf6f0 con 0x7fb04c1673d0 2024-09-12T01:34:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.806+0000 7fb04affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb04c15e760 0x7fb04c0cefe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56058/0 (socket says 172.21.15.16:56058) 2024-09-12T01:34:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.806+0000 7fb04affd640 1 -- 172.21.15.16:0/43581513 learned_addr learned my addr 172.21.15.16:0/43581513 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.806+0000 7fb04bfff640 1 --2- 172.21.15.16:0/43581513 >> v2:172.21.15.33:3300/0 conn(0x7fb04c1673d0 0x7fb04c1697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.806+0000 7fb04a7fc640 1 -- 172.21.15.16:0/43581513 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1752285983 0 0) 0x7fb04c073ec0 con 0x7fb04c15e760 2024-09-12T01:34:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.806+0000 7fb04a7fc640 1 -- 172.21.15.16:0/43581513 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb034003520 con 0x7fb04c15e760 2024-09-12T01:34:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.806+0000 7fb04a7fc640 1 -- 172.21.15.16:0/43581513 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2964669606 0 0) 0x7fb04c072e20 con 0x7fb04c06baf0 2024-09-12T01:34:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.806+0000 7fb04a7fc640 1 -- 172.21.15.16:0/43581513 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb04c073ec0 con 0x7fb04c06baf0 2024-09-12T01:34:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.806+0000 7fb04bfff640 1 -- 172.21.15.16:0/43581513 >> v1:172.21.15.33:6789/0 conn(0x7fb04c06baf0 legacy=0x7fb04c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.806+0000 7fb04bfff640 1 -- 172.21.15.16:0/43581513 >> v1:172.21.15.16:6789/0 conn(0x7fb04c15e760 legacy=0x7fb04c0cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.806+0000 7fb04a7fc640 1 -- 172.21.15.16:0/43581513 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1373760386 0 0) 0x7fb034003520 con 0x7fb04c15e760 2024-09-12T01:34:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.806+0000 7fb04bfff640 1 -- 172.21.15.16:0/43581513 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04c073100 con 0x7fb04c1673d0 2024-09-12T01:34:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.807+0000 7fb04bfff640 1 --2- 172.21.15.16:0/43581513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c1673d0 0x7fb04c1697c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb044009d90 tx=0x7fb04402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b03a3b5c3d37fe8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.807+0000 7fb04a7fc640 1 -- 172.21.15.16:0/43581513 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040001670 con 0x7fb04c1673d0 2024-09-12T01:34:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.807+0000 7fb04a7fc640 1 -- 172.21.15.16:0/43581513 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb044007540 con 0x7fb04c1673d0 2024-09-12T01:34:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.807+0000 7fb04a7fc640 1 -- 172.21.15.16:0/43581513 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0440078f0 con 0x7fb04c1673d0 2024-09-12T01:34:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.808+0000 7fb051bcf640 1 -- 172.21.15.16:0/43581513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c1673d0 msgr2=0x7fb04c1697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.808+0000 7fb051bcf640 1 --2- 172.21.15.16:0/43581513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c1673d0 0x7fb04c1697c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb044009d90 tx=0x7fb04402a5f0 comp rx=0 tx=0).stop 2024-09-12T01:34:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.808+0000 7fb051bcf640 1 -- 172.21.15.16:0/43581513 shutdown_connections 2024-09-12T01:34:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.808+0000 7fb051bcf640 1 --2- 172.21.15.16:0/43581513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c1673d0 0x7fb04c1697c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.808+0000 7fb051bcf640 1 -- 172.21.15.16:0/43581513 >> 172.21.15.16:0/43581513 conn(0x7fb04c0b2c50 msgr2=0x7fb04c0b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.808+0000 7fb051bcf640 1 -- 172.21.15.16:0/43581513 shutdown_connections 2024-09-12T01:34:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.808+0000 7fb051bcf640 1 -- 172.21.15.16:0/43581513 wait complete. 2024-09-12T01:34:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.809+0000 7fb051bcf640 1 Processor -- start 2024-09-12T01:34:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.809+0000 7fb051bcf640 1 -- start start 2024-09-12T01:34:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb051bcf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c06baf0 0x7fb04c0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb051bcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c15e760 0x7fb04c159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb051bcf640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb04c1673d0 0x7fb04c156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb051bcf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c0739a0 con 0x7fb04c15e760 2024-09-12T01:34:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb051bcf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c0736c0 con 0x7fb04c06baf0 2024-09-12T01:34:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb051bcf640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb04c06c850 con 0x7fb04c1673d0 2024-09-12T01:34:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb04affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c15e760 0x7fb04c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb04affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c15e760 0x7fb04c159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36392/0 (socket says 172.21.15.16:36392) 2024-09-12T01:34:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb04affd640 1 -- 172.21.15.16:0/894441345 learned_addr learned my addr 172.21.15.16:0/894441345 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb04b7fe640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c06baf0 0x7fb04c0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb04bfff640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb04c1673d0 0x7fb04c156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb04affd640 1 -- 172.21.15.16:0/894441345 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb04c1673d0 msgr2=0x7fb04c156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb04affd640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb04c1673d0 0x7fb04c156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb04affd640 1 -- 172.21.15.16:0/894441345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c06baf0 msgr2=0x7fb04c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb04affd640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c06baf0 0x7fb04c0ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.810+0000 7fb04affd640 1 -- 172.21.15.16:0/894441345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04c163270 con 0x7fb04c15e760 2024-09-12T01:34:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.811+0000 7fb04affd640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c15e760 0x7fb04c159620 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb040000fd0 tx=0x7fb04000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.811+0000 7fb033fff640 1 -- 172.21.15.16:0/894441345 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040017070 con 0x7fb04c15e760 2024-09-12T01:34:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.811+0000 7fb033fff640 1 -- 172.21.15.16:0/894441345 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb040005ce0 con 0x7fb04c15e760 2024-09-12T01:34:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.811+0000 7fb033fff640 1 -- 172.21.15.16:0/894441345 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0400072b0 con 0x7fb04c15e760 2024-09-12T01:34:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.811+0000 7fb051bcf640 1 -- 172.21.15.16:0/894441345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04c160880 con 0x7fb04c15e760 2024-09-12T01:34:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.811+0000 7fb051bcf640 1 -- 172.21.15.16:0/894441345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb04c163510 con 0x7fb04c15e760 2024-09-12T01:34:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.812+0000 7fb051bcf640 1 -- 172.21.15.16:0/894441345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb04c163710 con 0x7fb04c15e760 2024-09-12T01:34:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.812+0000 7fb033fff640 1 -- 172.21.15.16:0/894441345 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb04001a070 con 0x7fb04c15e760 2024-09-12T01:34:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.813+0000 7fb033fff640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb03806be30 0x7fb03806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.813+0000 7fb04b7fe640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb03806be30 0x7fb03806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.813+0000 7fb033fff640 1 -- 172.21.15.16:0/894441345 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7fb04008a350 con 0x7fb04c15e760 2024-09-12T01:34:33.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.814+0000 7fb04b7fe640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb03806be30 0x7fb03806e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0011e0 tx=0x7fb03c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:33.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:33.818+0000 7fb033fff640 1 -- 172.21.15.16:0/894441345 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb04c163710 con 0x7fb04c15e760 2024-09-12T01:34:34.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.030+0000 7fb051bcf640 1 -- 172.21.15.16:0/894441345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb04c06bed0 con 0x7fb04c15e760 2024-09-12T01:34:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.030+0000 7fb033fff640 1 -- 172.21.15.16:0/894441345 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5215 (secure 0 0 0) 0x7fb0400584c0 con 0x7fb04c15e760 2024-09-12T01:34:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:34.030 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"btime":"2024-09-12T01:34:33:188653+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/842803487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":842803487},{"type":"v1","addr":"172.21.15.33:6833","nonce":842803487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/70052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":70052721},{"type":"v1","addr":"172.21.15.33:6835","nonce":70052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3777111462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3777111462},{"type":"v1","addr":"172.21.15.16:6837","nonce":3777111462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:34:06.918906+0000","modified":"2024-09-12T01:34:32.191900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"a","rank":0,"incarnation":18,"state":"up:clientreplay","state_seq":11,"addr":"172.21.15.16:6835/2095784417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2095784417},{"type":"v1","addr":"172.21.15.16:6835","nonce":2095784417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-12T01:34:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2024-09-12T01:34:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.033+0000 7fb051bcf640 1 -- 172.21.15.16:0/894441345 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb03806be30 msgr2=0x7fb03806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.033+0000 7fb051bcf640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb03806be30 0x7fb03806e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0011e0 tx=0x7fb03c009040 comp rx=0 tx=0).stop 2024-09-12T01:34:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.033+0000 7fb051bcf640 1 -- 172.21.15.16:0/894441345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c15e760 msgr2=0x7fb04c159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.033+0000 7fb051bcf640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c15e760 0x7fb04c159620 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb040000fd0 tx=0x7fb04000c9b0 comp rx=0 tx=0).stop 2024-09-12T01:34:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.033+0000 7fb051bcf640 1 -- 172.21.15.16:0/894441345 shutdown_connections 2024-09-12T01:34:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.033+0000 7fb051bcf640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb03806be30 0x7fb03806e2f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.033+0000 7fb051bcf640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb04c1673d0 0x7fb04c156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.033+0000 7fb051bcf640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c15e760 0x7fb04c159620 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.033+0000 7fb051bcf640 1 --2- 172.21.15.16:0/894441345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c06baf0 0x7fb04c0ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.033+0000 7fb051bcf640 1 -- 172.21.15.16:0/894441345 >> 172.21.15.16:0/894441345 conn(0x7fb04c0b2c50 msgr2=0x7fb04c0744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:34.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.033+0000 7fb051bcf640 1 -- 172.21.15.16:0/894441345 shutdown_connections 2024-09-12T01:34:34.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:34.033+0000 7fb051bcf640 1 -- 172.21.15.16:0/894441345 wait complete. 2024-09-12T01:34:34.447 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T01:34:06.918906+0000', 'modified': '2024-09-12T01:34:32.191900+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4929}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4929': {'gid': 4929, 'name': 'a', 'rank': 0, 'incarnation': 18, 'state': 'up:clientreplay', 'state_seq': 11, 'addr': '172.21.15.16:6835/2095784417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 2095784417}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2095784417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-12T01:34:34.447 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4929:up:clientreplay 2024-09-12T01:34:35.448 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:34:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.565+0000 7f2c91027640 1 Processor -- start 2024-09-12T01:34:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.565+0000 7f2c91027640 1 -- start start 2024-09-12T01:34:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.566+0000 7f2c91027640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2c8c15e800 0x7f2c8c0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.566+0000 7f2c91027640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c8c0cf910 0x7f2c8c06d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.566+0000 7f2c91027640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c8c0cfd30 con 0x7f2c8c06dbd0 2024-09-12T01:34:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.566+0000 7f2c91027640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2c8c15ebe0 con 0x7f2c8c0cf910 2024-09-12T01:34:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.566+0000 7f2c91027640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8c0d42d0 con 0x7f2c8c15e800 2024-09-12T01:34:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.566+0000 7f2c8a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c8c0cf910 0x7f2c8c06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.566+0000 7f2c8a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c8c0cf910 0x7f2c8c06d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36396/0 (socket says 172.21.15.16:36396) 2024-09-12T01:34:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.566+0000 7f2c8a59c640 1 -- 172.21.15.16:0/812743957 learned_addr learned my addr 172.21.15.16:0/812743957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.566+0000 7f2c8ad9d640 1 --2- 172.21.15.16:0/812743957 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2c8c15e800 0x7f2c8c0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.566+0000 7f2c8a59c640 1 -- 172.21.15.16:0/812743957 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2c8c15e800 msgr2=0x7f2c8c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.566+0000 7f2c8a59c640 1 --2- 172.21.15.16:0/812743957 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2c8c15e800 0x7f2c8c0cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.566+0000 7f2c8a59c640 1 -- 172.21.15.16:0/812743957 >> v1:172.21.15.33:6789/0 conn(0x7f2c8c06dbd0 legacy=0x7f2c8c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.566+0000 7f2c8a59c640 1 -- 172.21.15.16:0/812743957 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c0d3ff0 con 0x7f2c8c0cf910 2024-09-12T01:34:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.567+0000 7f2c8a59c640 1 --2- 172.21.15.16:0/812743957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c8c0cf910 0x7f2c8c06d690 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009870 tx=0x7f2c8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3b2498b6d44575 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.567+0000 7f2c89d9b640 1 -- 172.21.15.16:0/812743957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80037070 con 0x7f2c8c0cf910 2024-09-12T01:34:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.567+0000 7f2c89d9b640 1 -- 172.21.15.16:0/812743957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c80032440 con 0x7f2c8c0cf910 2024-09-12T01:34:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.567+0000 7f2c89d9b640 1 -- 172.21.15.16:0/812743957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80032810 con 0x7f2c8c0cf910 2024-09-12T01:34:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.568+0000 7f2c91027640 1 -- 172.21.15.16:0/812743957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c8c0cf910 msgr2=0x7f2c8c06d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.568+0000 7f2c91027640 1 --2- 172.21.15.16:0/812743957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c8c0cf910 0x7f2c8c06d690 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009870 tx=0x7f2c8002a270 comp rx=0 tx=0).stop 2024-09-12T01:34:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.568+0000 7f2c91027640 1 -- 172.21.15.16:0/812743957 shutdown_connections 2024-09-12T01:34:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.568+0000 7f2c91027640 1 --2- 172.21.15.16:0/812743957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c8c0cf910 0x7f2c8c06d690 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.568+0000 7f2c91027640 1 --2- 172.21.15.16:0/812743957 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2c8c15e800 0x7f2c8c0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.568+0000 7f2c91027640 1 -- 172.21.15.16:0/812743957 >> 172.21.15.16:0/812743957 conn(0x7f2c8c09d750 msgr2=0x7f2c8c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.568+0000 7f2c91027640 1 -- 172.21.15.16:0/812743957 shutdown_connections 2024-09-12T01:34:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.568+0000 7f2c91027640 1 -- 172.21.15.16:0/812743957 wait complete. 2024-09-12T01:34:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.570+0000 7f2c91027640 1 Processor -- start 2024-09-12T01:34:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.570+0000 7f2c91027640 1 -- start start 2024-09-12T01:34:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.570+0000 7f2c91027640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c06dbd0 0x7f2c8c157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.570+0000 7f2c91027640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c8c0cf910 0x7f2c8c0ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.570+0000 7f2c91027640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2c8c15e800 0x7f2c8c0b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.570+0000 7f2c91027640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c06d170 con 0x7f2c8c0cf910 2024-09-12T01:34:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.570+0000 7f2c91027640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c06ce90 con 0x7f2c8c06dbd0 2024-09-12T01:34:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.570+0000 7f2c91027640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8c0d4e30 con 0x7f2c8c15e800 2024-09-12T01:34:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.570+0000 7f2c8a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c8c0cf910 0x7f2c8c0ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c8a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c8c0cf910 0x7f2c8c0ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36410/0 (socket says 172.21.15.16:36410) 2024-09-12T01:34:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c8ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c06dbd0 0x7f2c8c157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c8a59c640 1 -- 172.21.15.16:0/2392954493 learned_addr learned my addr 172.21.15.16:0/2392954493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c8b59e640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2c8c15e800 0x7f2c8c0b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c8a59c640 1 -- 172.21.15.16:0/2392954493 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2c8c15e800 msgr2=0x7f2c8c0b7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c8a59c640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2c8c15e800 0x7f2c8c0b7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c8a59c640 1 -- 172.21.15.16:0/2392954493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c06dbd0 msgr2=0x7f2c8c157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c8a59c640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c06dbd0 0x7f2c8c157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c8a59c640 1 -- 172.21.15.16:0/2392954493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c0cba30 con 0x7f2c8c0cf910 2024-09-12T01:34:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c8ad9d640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c06dbd0 0x7f2c8c157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c8a59c640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c8c0cf910 0x7f2c8c0ce270 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009870 tx=0x7f2c800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c737fe640 1 -- 172.21.15.16:0/2392954493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80041070 con 0x7f2c8c0cf910 2024-09-12T01:34:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c737fe640 1 -- 172.21.15.16:0/2392954493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c8003c890 con 0x7f2c8c0cf910 2024-09-12T01:34:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c91027640 1 -- 172.21.15.16:0/2392954493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8c0cbd10 con 0x7f2c8c0cf910 2024-09-12T01:34:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c737fe640 1 -- 172.21.15.16:0/2392954493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80037040 con 0x7f2c8c0cf910 2024-09-12T01:34:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.571+0000 7f2c91027640 1 -- 172.21.15.16:0/2392954493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8c0cc2d0 con 0x7f2c8c0cf910 2024-09-12T01:34:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.572+0000 7f2c91027640 1 -- 172.21.15.16:0/2392954493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c8c068880 con 0x7f2c8c0cf910 2024-09-12T01:34:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.573+0000 7f2c737fe640 1 -- 172.21.15.16:0/2392954493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c80044070 con 0x7f2c8c0cf910 2024-09-12T01:34:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.573+0000 7f2c737fe640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2c7806be60 0x7f2c7806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.574+0000 7f2c737fe640 1 -- 172.21.15.16:0/2392954493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f2c800ab980 con 0x7f2c8c0cf910 2024-09-12T01:34:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.574+0000 7f2c8ad9d640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2c7806be60 0x7f2c7806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.574+0000 7f2c8ad9d640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2c7806be60 0x7f2c7806e320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c007d90 tx=0x7f2c7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.578+0000 7f2c737fe640 1 -- 172.21.15.16:0/2392954493 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2c80079cb0 con 0x7f2c8c0cf910 2024-09-12T01:34:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.790+0000 7f2c91027640 1 -- 172.21.15.16:0/2392954493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c8c06d690 con 0x7f2c8c0cf910 2024-09-12T01:34:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.791+0000 7f2c737fe640 1 -- 172.21.15.16:0/2392954493 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5216 (secure 0 0 0) 0x7f2c8007dfb0 con 0x7f2c8c0cf910 2024-09-12T01:34:35.791 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:35.791 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"btime":"2024-09-12T01:34:34:193390+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/842803487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":842803487},{"type":"v1","addr":"172.21.15.33:6833","nonce":842803487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/70052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":70052721},{"type":"v1","addr":"172.21.15.33:6835","nonce":70052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3777111462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3777111462},{"type":"v1","addr":"172.21.15.16:6837","nonce":3777111462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:34:06.918906+0000","modified":"2024-09-12T01:34:34.193385+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.16:6835/2095784417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2095784417},{"type":"v1","addr":"172.21.15.16:6835","nonce":2095784417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4929,"qdb_cluster":[4929]},"id":2}]} 2024-09-12T01:34:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 22 2024-09-12T01:34:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.793+0000 7f2c91027640 1 -- 172.21.15.16:0/2392954493 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2c7806be60 msgr2=0x7f2c7806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.793+0000 7f2c91027640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2c7806be60 0x7f2c7806e320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c007d90 tx=0x7f2c7c008040 comp rx=0 tx=0).stop 2024-09-12T01:34:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.793+0000 7f2c91027640 1 -- 172.21.15.16:0/2392954493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c8c0cf910 msgr2=0x7f2c8c0ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.793+0000 7f2c91027640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c8c0cf910 0x7f2c8c0ce270 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009870 tx=0x7f2c800046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.793+0000 7f2c91027640 1 -- 172.21.15.16:0/2392954493 shutdown_connections 2024-09-12T01:34:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.793+0000 7f2c91027640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2c7806be60 0x7f2c7806e320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.793+0000 7f2c91027640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2c8c15e800 0x7f2c8c0b7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.793+0000 7f2c91027640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c8c0cf910 0x7f2c8c0ce270 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.793+0000 7f2c91027640 1 --2- 172.21.15.16:0/2392954493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c06dbd0 0x7f2c8c157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.793+0000 7f2c91027640 1 -- 172.21.15.16:0/2392954493 >> 172.21.15.16:0/2392954493 conn(0x7f2c8c09d750 msgr2=0x7f2c8c0b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.793+0000 7f2c91027640 1 -- 172.21.15.16:0/2392954493 shutdown_connections 2024-09-12T01:34:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:35.793+0000 7f2c91027640 1 -- 172.21.15.16:0/2392954493 wait complete. 2024-09-12T01:34:36.207 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T01:34:06.918906+0000', 'modified': '2024-09-12T01:34:34.193385+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4929}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4929': {'gid': 4929, 'name': 'a', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.16:6835/2095784417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 2095784417}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2095784417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4929, 'qdb_cluster': [4929]} 2024-09-12T01:34:36.207 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T01:34:36.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-12T01:34:36.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.316+0000 7f13cd70a640 1 Processor -- start 2024-09-12T01:34:36.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.316+0000 7f13cd70a640 1 -- start start 2024-09-12T01:34:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.316+0000 7f13cd70a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13c80d4d60 0x7f13c80d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.316+0000 7f13cd70a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f13c8162020 0x7f13c80d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.316+0000 7f13cd70a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c8162500 con 0x7f13c815e660 2024-09-12T01:34:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.316+0000 7f13cd70a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f13c80d2020 con 0x7f13c80d4d60 2024-09-12T01:34:36.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.316+0000 7f13cd70a640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f13c81627e0 con 0x7f13c8162020 2024-09-12T01:34:36.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.317+0000 7f13c67fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13c80d4d60 0x7f13c80d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.317+0000 7f13c67fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13c80d4d60 0x7f13c80d24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43166/0 (socket says 172.21.15.16:43166) 2024-09-12T01:34:36.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.317+0000 7f13c6ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f13c815e660 0x7f13c80d4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58196/0 (socket says 172.21.15.16:58196) 2024-09-12T01:34:36.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.317+0000 7f13c67fc640 1 -- 172.21.15.16:0/1041067978 learned_addr learned my addr 172.21.15.16:0/1041067978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:36.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.317+0000 7f13c77fe640 1 --2- 172.21.15.16:0/1041067978 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f13c8162020 0x7f13c80d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.317+0000 7f13c67fc640 1 -- 172.21.15.16:0/1041067978 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f13c8162020 msgr2=0x7f13c80d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:36.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.317+0000 7f13c67fc640 1 --2- 172.21.15.16:0/1041067978 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f13c8162020 0x7f13c80d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.317+0000 7f13c67fc640 1 -- 172.21.15.16:0/1041067978 >> v1:172.21.15.16:6789/0 conn(0x7f13c815e660 legacy=0x7f13c80d4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.317+0000 7f13c67fc640 1 -- 172.21.15.16:0/1041067978 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c8162ac0 con 0x7f13c80d4d60 2024-09-12T01:34:36.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.318+0000 7f13c67fc640 1 --2- 172.21.15.16:0/1041067978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c80d4d60 0x7f13c80d24b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009870 tx=0x7f13b002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a68d92a69703496 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:36.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.318+0000 7f13c5ffb640 1 -- 172.21.15.16:0/1041067978 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0037070 con 0x7f13c80d4d60 2024-09-12T01:34:36.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.318+0000 7f13c5ffb640 1 -- 172.21.15.16:0/1041067978 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b0032440 con 0x7f13c80d4d60 2024-09-12T01:34:36.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.318+0000 7f13c5ffb640 1 -- 172.21.15.16:0/1041067978 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0032810 con 0x7f13c80d4d60 2024-09-12T01:34:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.318+0000 7f13cd70a640 1 -- 172.21.15.16:0/1041067978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c80d4d60 msgr2=0x7f13c80d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.318+0000 7f13cd70a640 1 --2- 172.21.15.16:0/1041067978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c80d4d60 0x7f13c80d24b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009870 tx=0x7f13b002a1c0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.318+0000 7f13cd70a640 1 -- 172.21.15.16:0/1041067978 shutdown_connections 2024-09-12T01:34:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.318+0000 7f13cd70a640 1 --2- 172.21.15.16:0/1041067978 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f13c8162020 0x7f13c80d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.318+0000 7f13cd70a640 1 --2- 172.21.15.16:0/1041067978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c80d4d60 0x7f13c80d24b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.318+0000 7f13cd70a640 1 -- 172.21.15.16:0/1041067978 >> 172.21.15.16:0/1041067978 conn(0x7f13c808de10 msgr2=0x7f13c809d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.318+0000 7f13cd70a640 1 -- 172.21.15.16:0/1041067978 shutdown_connections 2024-09-12T01:34:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.319+0000 7f13cd70a640 1 -- 172.21.15.16:0/1041067978 wait complete. 2024-09-12T01:34:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.320+0000 7f13cd70a640 1 Processor -- start 2024-09-12T01:34:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.320+0000 7f13cd70a640 1 -- start start 2024-09-12T01:34:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13cd70a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13c80d4d60 0x7f13c81e7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13cd70a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c815e660 0x7f13c81e9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13cd70a640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f13c8162020 0x7f13c8156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13cd70a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13c80d4990 con 0x7f13c815e660 2024-09-12T01:34:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13cd70a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13c80d4810 con 0x7f13c80d4d60 2024-09-12T01:34:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13cd70a640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f13c80d4b10 con 0x7f13c8162020 2024-09-12T01:34:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13c67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c815e660 0x7f13c81e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13c67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c815e660 0x7f13c81e9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43176/0 (socket says 172.21.15.16:43176) 2024-09-12T01:34:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13c67fc640 1 -- 172.21.15.16:0/2979780758 learned_addr learned my addr 172.21.15.16:0/2979780758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13c77fe640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f13c8162020 0x7f13c8156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13c6ffd640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13c80d4d60 0x7f13c81e7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13c67fc640 1 -- 172.21.15.16:0/2979780758 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f13c8162020 msgr2=0x7f13c8156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13c67fc640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f13c8162020 0x7f13c8156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13c67fc640 1 -- 172.21.15.16:0/2979780758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13c80d4d60 msgr2=0x7f13c81e7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13c67fc640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13c80d4d60 0x7f13c81e7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13c67fc640 1 -- 172.21.15.16:0/2979780758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c81e8470 con 0x7f13c815e660 2024-09-12T01:34:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.321+0000 7f13c67fc640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c815e660 0x7f13c81e9510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f13b002a200 tx=0x7f13b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.322+0000 7f13bf7fe640 1 -- 172.21.15.16:0/2979780758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0041020 con 0x7f13c815e660 2024-09-12T01:34:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.322+0000 7f13bf7fe640 1 -- 172.21.15.16:0/2979780758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b00306c0 con 0x7f13c815e660 2024-09-12T01:34:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.322+0000 7f13cd70a640 1 -- 172.21.15.16:0/2979780758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c81e8750 con 0x7f13c815e660 2024-09-12T01:34:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.322+0000 7f13bf7fe640 1 -- 172.21.15.16:0/2979780758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0037040 con 0x7f13c815e660 2024-09-12T01:34:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.322+0000 7f13cd70a640 1 -- 172.21.15.16:0/2979780758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c81e8d10 con 0x7f13c815e660 2024-09-12T01:34:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.323+0000 7f13bf7fe640 1 -- 172.21.15.16:0/2979780758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f13b0044020 con 0x7f13c815e660 2024-09-12T01:34:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.324+0000 7f13bf7fe640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f13a806be60 0x7f13a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.324+0000 7f13bf7fe640 1 -- 172.21.15.16:0/2979780758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f13b00ab7d0 con 0x7f13c815e660 2024-09-12T01:34:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.324+0000 7f13c6ffd640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f13a806be60 0x7f13a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.324+0000 7f13c6ffd640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f13a806be60 0x7f13a806e320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f13b80012e0 tx=0x7f13b8005b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.325+0000 7f13d04a2740 1 Processor -- start 2024-09-12T01:34:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.325+0000 7f13d04a2740 1 -- start start 2024-09-12T01:34:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13d04a2740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342df2c0 0x560f34324a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13d04a2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342dca80 0x560f342df6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13d04a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342e0a50 0x560f342d98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13d04a2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f34241520 con 0x560f342dca80 2024-09-12T01:34:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13d04a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560f341e93e0 con 0x560f342e0a50 2024-09-12T01:34:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13d04a2740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x560f3423d4e0 con 0x560f342df2c0 2024-09-12T01:34:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13c67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342dca80 0x560f342df6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13c67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342dca80 0x560f342df6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43180/0 (socket says 172.21.15.16:43180) 2024-09-12T01:34:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13c67fc640 1 -- 172.21.15.16:0/3670783173 learned_addr learned my addr 172.21.15.16:0/3670783173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13c6ffd640 1 --2- 172.21.15.16:0/3670783173 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342df2c0 0x560f34324a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13c77fe640 1 --2- 172.21.15.16:0/3670783173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342e0a50 0x560f342d98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13c67fc640 1 -- 172.21.15.16:0/3670783173 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342df2c0 msgr2=0x560f34324a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13c67fc640 1 --2- 172.21.15.16:0/3670783173 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342df2c0 0x560f34324a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13c67fc640 1 -- 172.21.15.16:0/3670783173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342e0a50 msgr2=0x560f342d98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13c67fc640 1 --2- 172.21.15.16:0/3670783173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342e0a50 0x560f342d98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13c67fc640 1 -- 172.21.15.16:0/3670783173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f342042f0 con 0x560f342dca80 2024-09-12T01:34:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13c67fc640 1 --2- 172.21.15.16:0/3670783173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342dca80 0x560f342df6a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f13b00a9b00 tx=0x7f13b0079330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c528c38d59dd087b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.326+0000 7f13bcff9640 1 -- 172.21.15.16:0/3670783173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b007dcc0 con 0x560f342dca80 2024-09-12T01:34:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.327+0000 7f13bcff9640 1 -- 172.21.15.16:0/3670783173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b007ee50 con 0x560f342dca80 2024-09-12T01:34:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.327+0000 7f13bcff9640 1 -- 172.21.15.16:0/3670783173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b00824f0 con 0x560f342dca80 2024-09-12T01:34:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.327+0000 7f13d04a2740 1 -- 172.21.15.16:0/3670783173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342dca80 msgr2=0x560f342df6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.327+0000 7f13d04a2740 1 --2- 172.21.15.16:0/3670783173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342dca80 0x560f342df6a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f13b00a9b00 tx=0x7f13b0079330 comp rx=0 tx=0).stop 2024-09-12T01:34:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.327+0000 7f13d04a2740 1 -- 172.21.15.16:0/3670783173 shutdown_connections 2024-09-12T01:34:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.327+0000 7f13d04a2740 1 --2- 172.21.15.16:0/3670783173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342e0a50 0x560f342d98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.327+0000 7f13d04a2740 1 --2- 172.21.15.16:0/3670783173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342dca80 0x560f342df6a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.327+0000 7f13d04a2740 1 --2- 172.21.15.16:0/3670783173 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342df2c0 0x560f34324a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.327+0000 7f13d04a2740 1 -- 172.21.15.16:0/3670783173 >> 172.21.15.16:0/3670783173 conn(0x560f342dba80 msgr2=0x560f34323f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.327+0000 7f13d04a2740 1 -- 172.21.15.16:0/3670783173 shutdown_connections 2024-09-12T01:34:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.327+0000 7f13d04a2740 1 -- 172.21.15.16:0/3670783173 wait complete. 2024-09-12T01:34:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.328+0000 7f13d04a2740 1 -- start start 2024-09-12T01:34:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.328+0000 7f13d04a2740 1 Processor -- start 2024-09-12T01:34:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13d04a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342df2c0 0x560f34337c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13d04a2740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342e0a50 0x560f342fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13d04a2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342fdcf0 0x560f342de340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13d04a2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f34241520 con 0x560f342fdcf0 2024-09-12T01:34:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13d04a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560f341ca450 con 0x560f342df2c0 2024-09-12T01:34:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13d04a2740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x560f342daf20 con 0x560f342e0a50 2024-09-12T01:34:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13d04a2740 10 client.0 _ll_register_callbacks cb 0x560f33f53ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342fdcf0 0x560f342de340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342fdcf0 0x560f342de340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43190/0 (socket says 172.21.15.16:43190) 2024-09-12T01:34:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13c77fe640 1 -- 172.21.15.16:0/3932284917 learned_addr learned my addr 172.21.15.16:0/3932284917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13c6ffd640 1 --2- 172.21.15.16:0/3932284917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342df2c0 0x560f34337c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13c67fc640 1 --2- 172.21.15.16:0/3932284917 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342e0a50 0x560f342fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13c77fe640 1 -- 172.21.15.16:0/3932284917 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342e0a50 msgr2=0x560f342fd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13c77fe640 1 --2- 172.21.15.16:0/3932284917 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342e0a50 0x560f342fd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13c77fe640 1 -- 172.21.15.16:0/3932284917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342df2c0 msgr2=0x560f34337c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13c77fe640 1 --2- 172.21.15.16:0/3932284917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342df2c0 0x560f34337c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13c77fe640 1 -- 172.21.15.16:0/3932284917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f33f542d0 con 0x560f342fdcf0 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13c77fe640 1 --2- 172.21.15.16:0/3932284917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342fdcf0 0x560f342de340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f13c000bf60 tx=0x7f13c000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e869d411d6200217 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.329+0000 7f13a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13a5ffb640 1 -- 172.21.15.16:0/3932284917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0019030 con 0x560f342fdcf0 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13a5ffb640 1 -- 172.21.15.16:0/3932284917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f13c000f040 con 0x560f342fdcf0 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13d04a2740 1 -- 172.21.15.16:0/3932284917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13b007ee50 con 0x560f342fdcf0 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13a5ffb640 1 -- 172.21.15.16:0/3932284917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f13c0005990 con 0x560f342fdcf0 2024-09-12T01:34:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13d04a2740 10 client.5199 fetch_fsmap learned FSMap version 22 2024-09-12T01:34:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13d04a2740 1 -- 172.21.15.16:0/3932284917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x560f3401e6b0 con 0x560f342fdcf0 2024-09-12T01:34:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13a5ffb640 1 -- 172.21.15.16:0/3932284917 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b007ee50 con 0x560f342fdcf0 2024-09-12T01:34:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13a5ffb640 1 -- 172.21.15.16:0/3932284917 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 22) ==== 3502+0+0 (secure 0 0 0) 0x7f13c0028080 con 0x560f342fdcf0 2024-09-12T01:34:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13d04a2740 10 client.5199 fetch_fsmap finished waiting for FSMap version 22 2024-09-12T01:34:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13d04a2740 10 client.5199 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T01:34:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13d04a2740 20 client.5199 populate_metadata read hostname 'smithi016' 2024-09-12T01:34:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13d04a2740 1 --2- 172.21.15.16:0/3932284917 >> [v2:172.21.15.16:6834/2095784417,v1:172.21.15.16:6835/2095784417] conn(0x560f342d88a0 0x560f342d8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13d04a2740 4 client.5199 mds_command: new command op to 4929 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:34:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13d04a2740 1 -- 172.21.15.16:0/3932284917 --> [v2:172.21.15.16:6834/2095784417,v1:172.21.15.16:6835/2095784417] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f3424ec50 con 0x560f342d88a0 2024-09-12T01:34:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.330+0000 7f13c6ffd640 1 --2- 172.21.15.16:0/3932284917 >> [v2:172.21.15.16:6834/2095784417,v1:172.21.15.16:6835/2095784417] conn(0x560f342d88a0 0x560f342d8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13a5ffb640 10 client.5199 ms_handle_connect on v2:172.21.15.16:6834/2095784417 2024-09-12T01:34:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13c6ffd640 1 --2- 172.21.15.16:0/3932284917 >> [v2:172.21.15.16:6834/2095784417,v1:172.21.15.16:6835/2095784417] conn(0x560f342d88a0 0x560f342d8c80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52149e3046e9e650 server_cookie=6e0537b7f906fc49 in_seq=0 out_seq=0 2024-09-12T01:34:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13a5ffb640 1 -- 172.21.15.16:0/3932284917 <== mds.0 v2:172.21.15.16:6834/2095784417 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f3424ec50 con 0x560f342d88a0 2024-09-12T01:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13a5ffb640 10 client.5199 handle_command_reply: tid=0 2024-09-12T01:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13d04a2740 1 client.5199 shutdown 2024-09-12T01:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13d04a2740 1 -- 172.21.15.16:0/3932284917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342fdcf0 msgr2=0x560f342de340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13d04a2740 1 --2- 172.21.15.16:0/3932284917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342fdcf0 0x560f342de340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f13c000bf60 tx=0x7f13c000bf90 comp rx=0 tx=0).stop 2024-09-12T01:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13d04a2740 1 -- 172.21.15.16:0/3932284917 shutdown_connections 2024-09-12T01:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13d04a2740 1 --2- 172.21.15.16:0/3932284917 >> [v2:172.21.15.16:6834/2095784417,v1:172.21.15.16:6835/2095784417] conn(0x560f342d88a0 0x560f342d8c80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13d04a2740 1 --2- 172.21.15.16:0/3932284917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342fdcf0 0x560f342de340 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13d04a2740 1 --2- 172.21.15.16:0/3932284917 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342e0a50 0x560f342fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13a5ffb640 0 client.5199 ms_handle_reset on v2:172.21.15.16:6834/2095784417 2024-09-12T01:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13d04a2740 1 --2- 172.21.15.16:0/3932284917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342df2c0 0x560f34337c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.331+0000 7f13d04a2740 1 -- 172.21.15.16:0/3932284917 >> 172.21.15.16:0/3932284917 conn(0x560f342dca80 msgr2=0x560f343234e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.332+0000 7f13d04a2740 1 -- 172.21.15.16:0/3932284917 shutdown_connections 2024-09-12T01:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.332+0000 7f13d04a2740 1 -- 172.21.15.16:0/3932284917 wait complete. 2024-09-12T01:34:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.332+0000 7f13d04a2740 20 client.5199 trim_cache size 0 max 16384 2024-09-12T01:34:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.350+0000 7f13d04a2740 1 Processor -- start 2024-09-12T01:34:36.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.350+0000 7f13d04a2740 1 -- start start 2024-09-12T01:34:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13d04a2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342df2c0 0x560f343371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13d04a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342e0a50 0x560f34337730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13d04a2740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342fdcf0 0x560f34337c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13d04a2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8200240 con 0x560f342df2c0 2024-09-12T01:34:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13d04a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13c81f83c0 con 0x560f342e0a50 2024-09-12T01:34:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13d04a2740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f13c81ffe40 con 0x560f342fdcf0 2024-09-12T01:34:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13c6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342df2c0 0x560f343371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13c6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342df2c0 0x560f343371f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43206/0 (socket says 172.21.15.16:43206) 2024-09-12T01:34:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13c6ffd640 1 -- 172.21.15.16:0/2874137704 learned_addr learned my addr 172.21.15.16:0/2874137704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13c77fe640 1 --2- 172.21.15.16:0/2874137704 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342fdcf0 0x560f34337c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13c67fc640 1 --2- 172.21.15.16:0/2874137704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342e0a50 0x560f34337730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13c6ffd640 1 -- 172.21.15.16:0/2874137704 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342fdcf0 msgr2=0x560f34337c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13c6ffd640 1 --2- 172.21.15.16:0/2874137704 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342fdcf0 0x560f34337c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13c6ffd640 1 -- 172.21.15.16:0/2874137704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342e0a50 msgr2=0x560f34337730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13c6ffd640 1 --2- 172.21.15.16:0/2874137704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342e0a50 0x560f34337730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13c6ffd640 1 -- 172.21.15.16:0/2874137704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f342da610 con 0x560f342df2c0 2024-09-12T01:34:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13c6ffd640 1 --2- 172.21.15.16:0/2874137704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342df2c0 0x560f343371f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f13b80018f0 tx=0x7f13b8009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfa805086130997 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13a7fff640 1 -- 172.21.15.16:0/2874137704 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8025930 con 0x560f342df2c0 2024-09-12T01:34:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13a7fff640 1 -- 172.21.15.16:0/2874137704 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b8025ad0 con 0x560f342df2c0 2024-09-12T01:34:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.351+0000 7f13a7fff640 1 -- 172.21.15.16:0/2874137704 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8025c80 con 0x560f342df2c0 2024-09-12T01:34:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.352+0000 7f13d04a2740 1 -- 172.21.15.16:0/2874137704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342df2c0 msgr2=0x560f343371f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.352+0000 7f13d04a2740 1 --2- 172.21.15.16:0/2874137704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342df2c0 0x560f343371f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f13b80018f0 tx=0x7f13b8009030 comp rx=0 tx=0).stop 2024-09-12T01:34:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.352+0000 7f13d04a2740 1 -- 172.21.15.16:0/2874137704 shutdown_connections 2024-09-12T01:34:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.352+0000 7f13d04a2740 1 --2- 172.21.15.16:0/2874137704 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342fdcf0 0x560f34337c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.352+0000 7f13d04a2740 1 --2- 172.21.15.16:0/2874137704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342e0a50 0x560f34337730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.352+0000 7f13d04a2740 1 --2- 172.21.15.16:0/2874137704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342df2c0 0x560f343371f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.352+0000 7f13d04a2740 1 -- 172.21.15.16:0/2874137704 >> 172.21.15.16:0/2874137704 conn(0x560f342dca80 msgr2=0x560f34322300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.352+0000 7f13d04a2740 1 -- 172.21.15.16:0/2874137704 shutdown_connections 2024-09-12T01:34:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.352+0000 7f13d04a2740 1 -- 172.21.15.16:0/2874137704 wait complete. 2024-09-12T01:34:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.353+0000 7f13d04a2740 1 -- start start 2024-09-12T01:34:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.353+0000 7f13d04a2740 1 Processor -- start 2024-09-12T01:34:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.353+0000 7f13d04a2740 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342df2c0 0x560f343044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.353+0000 7f13d04a2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342e0a50 0x560f343238a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.353+0000 7f13d04a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342fdcf0 0x560f34323de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.353+0000 7f13d04a2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f343004f0 con 0x560f342e0a50 2024-09-12T01:34:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.353+0000 7f13d04a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560f34307820 con 0x560f342fdcf0 2024-09-12T01:34:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.353+0000 7f13d04a2740 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x560f342fff90 con 0x560f342df2c0 2024-09-12T01:34:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.353+0000 7f13d04a2740 10 client.0 _ll_register_callbacks cb 0x7f13c000bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13c67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342e0a50 0x560f343238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13c67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342e0a50 0x560f343238a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43208/0 (socket says 172.21.15.16:43208) 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13c67fc640 1 -- 172.21.15.16:0/1185742756 learned_addr learned my addr 172.21.15.16:0/1185742756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13c6ffd640 1 --2- 172.21.15.16:0/1185742756 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342df2c0 0x560f343044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13c77fe640 1 --2- 172.21.15.16:0/1185742756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342fdcf0 0x560f34323de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13c67fc640 1 -- 172.21.15.16:0/1185742756 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342df2c0 msgr2=0x560f343044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13c67fc640 1 --2- 172.21.15.16:0/1185742756 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342df2c0 0x560f343044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13c67fc640 1 -- 172.21.15.16:0/1185742756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342fdcf0 msgr2=0x560f34323de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13c67fc640 1 --2- 172.21.15.16:0/1185742756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342fdcf0 0x560f34323de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13c67fc640 1 -- 172.21.15.16:0/1185742756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f343a2510 con 0x560f342e0a50 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13c67fc640 1 --2- 172.21.15.16:0/1185742756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342e0a50 0x560f343238a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f13b0079850 tx=0x7f13b00ab040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b16f4c9f3e77403 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13a5ffb640 1 -- 172.21.15.16:0/1185742756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0097070 con 0x560f342e0a50 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13a5ffb640 1 -- 172.21.15.16:0/1185742756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f13b0082060 con 0x560f342e0a50 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13d04a2740 1 -- 172.21.15.16:0/1185742756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13b8025ad0 con 0x560f342e0a50 2024-09-12T01:34:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13a5ffb640 1 -- 172.21.15.16:0/1185742756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f13b0075bb0 con 0x560f342e0a50 2024-09-12T01:34:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13d04a2740 10 client.5205 fetch_fsmap learned FSMap version 22 2024-09-12T01:34:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13d04a2740 1 -- 172.21.15.16:0/1185742756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x560f34308170 con 0x560f342e0a50 2024-09-12T01:34:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.354+0000 7f13a5ffb640 1 -- 172.21.15.16:0/1185742756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b8025ad0 con 0x560f342e0a50 2024-09-12T01:34:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13a5ffb640 1 -- 172.21.15.16:0/1185742756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 22) ==== 3502+0+0 (secure 0 0 0) 0x7f13b003a030 con 0x560f342e0a50 2024-09-12T01:34:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 10 client.5205 fetch_fsmap finished waiting for FSMap version 22 2024-09-12T01:34:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 10 client.5205 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T01:34:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 20 client.5205 populate_metadata read hostname 'smithi016' 2024-09-12T01:34:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 1 --2- 172.21.15.16:0/1185742756 >> [v2:172.21.15.16:6834/2095784417,v1:172.21.15.16:6835/2095784417] conn(0x560f342fe810 0x560f343a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 4 client.5205 mds_command: new command op to 4929 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:34:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 1 -- 172.21.15.16:0/1185742756 --> [v2:172.21.15.16:6834/2095784417,v1:172.21.15.16:6835/2095784417] -- command(tid 0: {"prefix": "status"}) -- 0x560f34248fa0 con 0x560f342fe810 2024-09-12T01:34:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13c6ffd640 1 --2- 172.21.15.16:0/1185742756 >> [v2:172.21.15.16:6834/2095784417,v1:172.21.15.16:6835/2095784417] conn(0x560f342fe810 0x560f343a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13c6ffd640 1 --2- 172.21.15.16:0/1185742756 >> [v2:172.21.15.16:6834/2095784417,v1:172.21.15.16:6835/2095784417] conn(0x560f342fe810 0x560f343a57b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a604a8fb5fd40109 server_cookie=5d50d4e7565167bc in_seq=0 out_seq=0 2024-09-12T01:34:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13a5ffb640 10 client.5205 ms_handle_connect on v2:172.21.15.16:6834/2095784417 2024-09-12T01:34:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13a5ffb640 1 -- 172.21.15.16:0/1185742756 <== mds.0 v2:172.21.15.16:6834/2095784417 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560f34248fa0 con 0x560f342fe810 2024-09-12T01:34:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13a5ffb640 10 client.5205 handle_command_reply: tid=0 2024-09-12T01:34:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 1 client.5205 shutdown 2024-09-12T01:34:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 1 -- 172.21.15.16:0/1185742756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342e0a50 msgr2=0x560f343238a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 1 --2- 172.21.15.16:0/1185742756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342e0a50 0x560f343238a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f13b0079850 tx=0x7f13b00ab040 comp rx=0 tx=0).stop 2024-09-12T01:34:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 1 -- 172.21.15.16:0/1185742756 shutdown_connections 2024-09-12T01:34:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 1 --2- 172.21.15.16:0/1185742756 >> [v2:172.21.15.16:6834/2095784417,v1:172.21.15.16:6835/2095784417] conn(0x560f342fe810 0x560f343a57b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 1 --2- 172.21.15.16:0/1185742756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f342fdcf0 0x560f34323de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 1 --2- 172.21.15.16:0/1185742756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f342e0a50 0x560f343238a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13a5ffb640 0 client.5205 ms_handle_reset on v2:172.21.15.16:6834/2095784417 2024-09-12T01:34:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 1 --2- 172.21.15.16:0/1185742756 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x560f342df2c0 0x560f343044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 1 -- 172.21.15.16:0/1185742756 >> 172.21.15.16:0/1185742756 conn(0x560f342dca80 msgr2=0x560f34327440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 1 -- 172.21.15.16:0/1185742756 shutdown_connections 2024-09-12T01:34:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.355+0000 7f13d04a2740 1 -- 172.21.15.16:0/1185742756 wait complete. 2024-09-12T01:34:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.356+0000 7f13d04a2740 20 client.5205 trim_cache size 0 max 16384 2024-09-12T01:34:36.362 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:34:36.362 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "cb5903e4-2f7f-485b-9d92-173a6afaa2f8", 2024-09-12T01:34:36.362 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-12T01:34:36.362 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4929, 2024-09-12T01:34:36.362 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T01:34:36.362 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T01:34:36.363 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T01:34:36.363 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.1828512299999998, 2024-09-12T01:34:36.363 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 22, 2024-09-12T01:34:36.363 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 41, 2024-09-12T01:34:36.363 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 41, 2024-09-12T01:34:36.363 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 35.658203802000003 2024-09-12T01:34:36.363 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:34:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.356+0000 7f13cd70a640 1 -- 172.21.15.16:0/2979780758 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f13a806be60 msgr2=0x7f13a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.356+0000 7f13cd70a640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f13a806be60 0x7f13a806e320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f13b80012e0 tx=0x7f13b8005b70 comp rx=0 tx=0).stop 2024-09-12T01:34:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.356+0000 7f13cd70a640 1 -- 172.21.15.16:0/2979780758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c815e660 msgr2=0x7f13c81e9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.356+0000 7f13cd70a640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c815e660 0x7f13c81e9510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f13b002a200 tx=0x7f13b00046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.356+0000 7f13cd70a640 1 -- 172.21.15.16:0/2979780758 shutdown_connections 2024-09-12T01:34:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.356+0000 7f13cd70a640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f13a806be60 0x7f13a806e320 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.356+0000 7f13cd70a640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f13c8162020 0x7f13c8156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.356+0000 7f13cd70a640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c815e660 0x7f13c81e9510 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.356+0000 7f13cd70a640 1 --2- 172.21.15.16:0/2979780758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13c80d4d60 0x7f13c81e7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.356+0000 7f13cd70a640 1 -- 172.21.15.16:0/2979780758 >> 172.21.15.16:0/2979780758 conn(0x7f13c808de10 msgr2=0x7f13c80cef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.356+0000 7f13cd70a640 1 -- 172.21.15.16:0/2979780758 shutdown_connections 2024-09-12T01:34:36.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.356+0000 7f13cd70a640 1 -- 172.21.15.16:0/2979780758 wait complete. 2024-09-12T01:34:36.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:36.367 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%a /home/ubuntu/cephtest/mnt.0/testdir/testfile) 2024-09-12T01:34:36.384 INFO:teuthology.orchestra.run.smithi016.stdout:600 2024-09-12T01:34:36.384 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T01:34:36.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T01:34:36.466 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T01:34:36.519 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T01:34:36.520 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T01:34:36.520 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:34:36.520 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:34:36.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T01:34:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.680+0000 7efe0d79f640 1 Processor -- start 2024-09-12T01:34:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.680+0000 7efe0d79f640 1 -- start start 2024-09-12T01:34:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.681+0000 7efe0d79f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe080d3090 0x7efe080cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.681+0000 7efe0d79f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7efe080cf460 0x7efe080c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.681+0000 7efe0d79f640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe080d2270 con 0x7efe0815e520 2024-09-12T01:34:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.681+0000 7efe0d79f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7efe080d07a0 con 0x7efe080d3090 2024-09-12T01:34:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.681+0000 7efe0d79f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7efe080cf8c0 con 0x7efe080cf460 2024-09-12T01:34:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.681+0000 7efe077fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe080d3090 0x7efe080cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.681+0000 7efe077fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe080d3090 0x7efe080cef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43214/0 (socket says 172.21.15.16:43214) 2024-09-12T01:34:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.681+0000 7efe077fe640 1 -- 172.21.15.16:0/3097796635 learned_addr learned my addr 172.21.15.16:0/3097796635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.682+0000 7efe0cf9e640 1 --2- 172.21.15.16:0/3097796635 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7efe080cf460 0x7efe080c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.682+0000 7efe077fe640 1 -- 172.21.15.16:0/3097796635 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7efe080cf460 msgr2=0x7efe080c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.682+0000 7efe077fe640 1 --2- 172.21.15.16:0/3097796635 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7efe080cf460 0x7efe080c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.682+0000 7efe077fe640 1 -- 172.21.15.16:0/3097796635 >> v1:172.21.15.33:6789/0 conn(0x7efe0815e520 legacy=0x7efe080d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.682+0000 7efe077fe640 1 -- 172.21.15.16:0/3097796635 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe080cfb90 con 0x7efe080d3090 2024-09-12T01:34:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.682+0000 7efe0cf9e640 1 --2- 172.21.15.16:0/3097796635 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7efe080cf460 0x7efe080c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.682+0000 7efe077fe640 1 --2- 172.21.15.16:0/3097796635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe080d3090 0x7efe080cef20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009870 tx=0x7efdfc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa89e10bda4372a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.683+0000 7efe06ffd640 1 -- 172.21.15.16:0/3097796635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc037070 con 0x7efe080d3090 2024-09-12T01:34:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.683+0000 7efe06ffd640 1 -- 172.21.15.16:0/3097796635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdfc032440 con 0x7efe080d3090 2024-09-12T01:34:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.683+0000 7efe06ffd640 1 -- 172.21.15.16:0/3097796635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc032810 con 0x7efe080d3090 2024-09-12T01:34:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.683+0000 7efe0d79f640 1 -- 172.21.15.16:0/3097796635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe080d3090 msgr2=0x7efe080cef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.683+0000 7efe0d79f640 1 --2- 172.21.15.16:0/3097796635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe080d3090 0x7efe080cef20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009870 tx=0x7efdfc02a270 comp rx=0 tx=0).stop 2024-09-12T01:34:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.683+0000 7efe0d79f640 1 -- 172.21.15.16:0/3097796635 shutdown_connections 2024-09-12T01:34:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.683+0000 7efe0d79f640 1 --2- 172.21.15.16:0/3097796635 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7efe080cf460 0x7efe080c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.683+0000 7efe0d79f640 1 --2- 172.21.15.16:0/3097796635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe080d3090 0x7efe080cef20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.683+0000 7efe0d79f640 1 -- 172.21.15.16:0/3097796635 >> 172.21.15.16:0/3097796635 conn(0x7efe0808dd50 msgr2=0x7efe0809d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.683+0000 7efe0d79f640 1 -- 172.21.15.16:0/3097796635 shutdown_connections 2024-09-12T01:34:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.683+0000 7efe0d79f640 1 -- 172.21.15.16:0/3097796635 wait complete. 2024-09-12T01:34:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.684+0000 7efe0d79f640 1 Processor -- start 2024-09-12T01:34:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.685+0000 7efe0d79f640 1 -- start start 2024-09-12T01:34:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.685+0000 7efe0d79f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7efe080cf460 0x7efe0809dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.685+0000 7efe0d79f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe080d3090 0x7efe081e7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.685+0000 7efe0d79f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe0815e520 0x7efe080b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.685+0000 7efe0d79f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe080d0430 con 0x7efe080d3090 2024-09-12T01:34:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.685+0000 7efe0d79f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe080d0150 con 0x7efe0815e520 2024-09-12T01:34:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.685+0000 7efe0d79f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7efe080c8670 con 0x7efe080cf460 2024-09-12T01:34:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.685+0000 7efe077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe080d3090 0x7efe081e7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.685+0000 7efe077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe080d3090 0x7efe081e7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43226/0 (socket says 172.21.15.16:43226) 2024-09-12T01:34:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.685+0000 7efe077fe640 1 -- 172.21.15.16:0/4139943730 learned_addr learned my addr 172.21.15.16:0/4139943730 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.685+0000 7efe0cf9e640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe0815e520 0x7efe080b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.685+0000 7efe07fff640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7efe080cf460 0x7efe0809dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.686+0000 7efe077fe640 1 -- 172.21.15.16:0/4139943730 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7efe080cf460 msgr2=0x7efe0809dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.686+0000 7efe077fe640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7efe080cf460 0x7efe0809dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.686+0000 7efe077fe640 1 -- 172.21.15.16:0/4139943730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe0815e520 msgr2=0x7efe080b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.686+0000 7efe077fe640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe0815e520 0x7efe080b7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.686+0000 7efe077fe640 1 -- 172.21.15.16:0/4139943730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe080cc380 con 0x7efe080d3090 2024-09-12T01:34:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.686+0000 7efe077fe640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe080d3090 0x7efe081e7be0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efdfc000c00 tx=0x7efdfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.686+0000 7efe04ff9640 1 -- 172.21.15.16:0/4139943730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc041020 con 0x7efe080d3090 2024-09-12T01:34:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.686+0000 7efe04ff9640 1 -- 172.21.15.16:0/4139943730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdfc0306c0 con 0x7efe080d3090 2024-09-12T01:34:36.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.686+0000 7efe0d79f640 1 -- 172.21.15.16:0/4139943730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe080ccd70 con 0x7efe080d3090 2024-09-12T01:34:36.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.686+0000 7efe04ff9640 1 -- 172.21.15.16:0/4139943730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc037040 con 0x7efe080d3090 2024-09-12T01:34:36.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.686+0000 7efe0d79f640 1 -- 172.21.15.16:0/4139943730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe080cd330 con 0x7efe080d3090 2024-09-12T01:34:36.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.687+0000 7efe0d79f640 1 -- 172.21.15.16:0/4139943730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe080c8130 con 0x7efe080d3090 2024-09-12T01:34:36.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.688+0000 7efe04ff9640 1 -- 172.21.15.16:0/4139943730 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efdfc044020 con 0x7efe080d3090 2024-09-12T01:34:36.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.688+0000 7efe04ff9640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7efdf406be30 0x7efdf406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:36.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.688+0000 7efe04ff9640 1 -- 172.21.15.16:0/4139943730 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7efdfc0ab6b0 con 0x7efe080d3090 2024-09-12T01:34:36.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.688+0000 7efe07fff640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7efdf406be30 0x7efdf406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:36.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.689+0000 7efe07fff640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7efdf406be30 0x7efdf406e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efdf80016e0 tx=0x7efdf8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:36.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.693+0000 7efe04ff9640 1 -- 172.21.15.16:0/4139943730 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efdfc04b030 con 0x7efe080d3090 2024-09-12T01:34:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.879+0000 7efe0d79f640 1 -- 172.21.15.16:0/4139943730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efe08069660 con 0x7efe080d3090 2024-09-12T01:34:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.880+0000 7efe04ff9640 1 -- 172.21.15.16:0/4139943730 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v41) ==== 80+0+122 (secure 0 0 0) 0x7efdfc0a9e20 con 0x7efe080d3090 2024-09-12T01:34:36.879 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6837/4214019228 2024-09-13T01:34:30.165507+0000 2024-09-12T01:34:36.880 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6836/4214019228 2024-09-13T01:34:30.165507+0000 2024-09-12T01:34:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:listed 2 entries 2024-09-12T01:34:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.882+0000 7efe0d79f640 1 -- 172.21.15.16:0/4139943730 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7efdf406be30 msgr2=0x7efdf406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.882+0000 7efe0d79f640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7efdf406be30 0x7efdf406e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efdf80016e0 tx=0x7efdf8009040 comp rx=0 tx=0).stop 2024-09-12T01:34:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.882+0000 7efe0d79f640 1 -- 172.21.15.16:0/4139943730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe080d3090 msgr2=0x7efe081e7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.882+0000 7efe0d79f640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe080d3090 0x7efe081e7be0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efdfc000c00 tx=0x7efdfc0046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.882+0000 7efe0d79f640 1 -- 172.21.15.16:0/4139943730 shutdown_connections 2024-09-12T01:34:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.882+0000 7efe0d79f640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7efdf406be30 0x7efdf406e2f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.882+0000 7efe0d79f640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe0815e520 0x7efe080b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.882+0000 7efe0d79f640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe080d3090 0x7efe081e7be0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.882+0000 7efe0d79f640 1 --2- 172.21.15.16:0/4139943730 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7efe080cf460 0x7efe0809dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.882+0000 7efe0d79f640 1 -- 172.21.15.16:0/4139943730 >> 172.21.15.16:0/4139943730 conn(0x7efe0808dd50 msgr2=0x7efe080b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.882+0000 7efe0d79f640 1 -- 172.21.15.16:0/4139943730 shutdown_connections 2024-09-12T01:34:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:36.882+0000 7efe0d79f640 1 -- 172.21.15.16:0/4139943730 wait complete. 2024-09-12T01:34:36.893 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-12T01:34:36.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:36.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-12T01:34:36.918 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[48210]: fuse finished with error 0 and tester_r 0 2024-09-12T01:34:36.920 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:34:36.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:36.920 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T01:34:36.994 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T01:34:36.994 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:34:36.994 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:34:37.014 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1991 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T01:34:37.015 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:34:37.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:37.015 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T01:34:37.074 INFO:teuthology.orchestra.run.smithi033.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-12T01:34:37.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:34:37.075 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T01:34:37.075 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:34:37.075 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:34:37.092 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=414 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T01:34:37.092 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:34:37.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:37.092 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-12T01:34:37.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:34:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-12T01:34:37.149 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T01:34:37.149 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:34:37.149 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:34:37.169 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1991 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T01:34:37.169 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:34:37.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:37.170 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-12T01:34:37.227 INFO:teuthology.orchestra.run.smithi033.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-12T01:34:37.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:34:37.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:34:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.332+0000 7ff643131640 1 Processor -- start 2024-09-12T01:34:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.332+0000 7ff643131640 1 -- start start 2024-09-12T01:34:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff643131640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff63c06dd90 0x7ff63c0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff643131640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff63c0cb580 0x7ff63c073720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff643131640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff63c06f530 con 0x7ff63c15e8d0 2024-09-12T01:34:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff643131640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7ff63c0cbbb0 con 0x7ff63c06dd90 2024-09-12T01:34:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff643131640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff63c0cbe90 con 0x7ff63c0cb580 2024-09-12T01:34:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff6416ce640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff63c0cb580 0x7ff63c073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff633fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff63c06dd90 0x7ff63c0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff6416ce640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff63c0cb580 0x7ff63c073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:47432/0 (socket says 172.21.15.16:47432) 2024-09-12T01:34:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff6416ce640 1 -- 172.21.15.16:0/190597162 learned_addr learned my addr 172.21.15.16:0/190597162 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff633fff640 1 -- 172.21.15.16:0/190597162 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff63c0cb580 msgr2=0x7ff63c073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff633fff640 1 --2- 172.21.15.16:0/190597162 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff63c0cb580 0x7ff63c073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff633fff640 1 -- 172.21.15.16:0/190597162 >> v1:172.21.15.33:6789/0 conn(0x7ff63c15e8d0 legacy=0x7ff63c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff6337fe640 1 -- 172.21.15.16:0/190597162 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3967627925 0 0) 0x7ff63c06f530 con 0x7ff63c15e8d0 2024-09-12T01:34:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.333+0000 7ff633fff640 1 -- 172.21.15.16:0/190597162 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c0ca5f0 con 0x7ff63c06dd90 2024-09-12T01:34:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.334+0000 7ff633fff640 1 --2- 172.21.15.16:0/190597162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c06dd90 0x7ff63c0acdd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0097a0 tx=0x7ff62c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=162e91213dba7097 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.334+0000 7ff6337fe640 1 -- 172.21.15.16:0/190597162 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c037070 con 0x7ff63c06dd90 2024-09-12T01:34:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.335+0000 7ff6337fe640 1 -- 172.21.15.16:0/190597162 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62c032440 con 0x7ff63c06dd90 2024-09-12T01:34:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.335+0000 7ff6337fe640 1 -- 172.21.15.16:0/190597162 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c032810 con 0x7ff63c06dd90 2024-09-12T01:34:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.335+0000 7ff643131640 1 -- 172.21.15.16:0/190597162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c06dd90 msgr2=0x7ff63c0acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.335+0000 7ff643131640 1 --2- 172.21.15.16:0/190597162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c06dd90 0x7ff63c0acdd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0097a0 tx=0x7ff62c02a180 comp rx=0 tx=0).stop 2024-09-12T01:34:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.335+0000 7ff643131640 1 -- 172.21.15.16:0/190597162 shutdown_connections 2024-09-12T01:34:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.335+0000 7ff643131640 1 --2- 172.21.15.16:0/190597162 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff63c0cb580 0x7ff63c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.335+0000 7ff643131640 1 --2- 172.21.15.16:0/190597162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c06dd90 0x7ff63c0acdd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.335+0000 7ff643131640 1 -- 172.21.15.16:0/190597162 >> 172.21.15.16:0/190597162 conn(0x7ff63c0cffa0 msgr2=0x7ff63c0d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.335+0000 7ff643131640 1 -- 172.21.15.16:0/190597162 shutdown_connections 2024-09-12T01:34:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.335+0000 7ff643131640 1 -- 172.21.15.16:0/190597162 wait complete. 2024-09-12T01:34:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.336+0000 7ff643131640 1 Processor -- start 2024-09-12T01:34:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.336+0000 7ff643131640 1 -- start start 2024-09-12T01:34:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.337+0000 7ff643131640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff63c06dd90 0x7ff63c148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.337+0000 7ff643131640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c0cb580 0x7ff63c09c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.337+0000 7ff643131640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff63c15e8d0 0x7ff63c06e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.337+0000 7ff643131640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c0ce860 con 0x7ff63c15e8d0 2024-09-12T01:34:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.337+0000 7ff643131640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c0cabb0 con 0x7ff63c0cb580 2024-09-12T01:34:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.337+0000 7ff643131640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff63c0ce9e0 con 0x7ff63c06dd90 2024-09-12T01:34:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.337+0000 7ff6416ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff63c15e8d0 0x7ff63c06e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.337+0000 7ff6416ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff63c15e8d0 0x7ff63c06e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43232/0 (socket says 172.21.15.16:43232) 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.337+0000 7ff6416ce640 1 -- 172.21.15.16:0/2581320542 learned_addr learned my addr 172.21.15.16:0/2581320542 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff633fff640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c0cb580 0x7ff63c09c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff640ecd640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff63c06dd90 0x7ff63c148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff6416ce640 1 -- 172.21.15.16:0/2581320542 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff63c06dd90 msgr2=0x7ff63c148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff6416ce640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff63c06dd90 0x7ff63c148d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff6416ce640 1 -- 172.21.15.16:0/2581320542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c0cb580 msgr2=0x7ff63c09c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff6416ce640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c0cb580 0x7ff63c09c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff6416ce640 1 -- 172.21.15.16:0/2581320542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c09d8e0 con 0x7ff63c15e8d0 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff633fff640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c0cb580 0x7ff63c09c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff640ecd640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff63c06dd90 0x7ff63c148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff6416ce640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff63c15e8d0 0x7ff63c06e300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff63400efc0 tx=0x7ff63400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff6317fa640 1 -- 172.21.15.16:0/2581320542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63400cd20 con 0x7ff63c15e8d0 2024-09-12T01:34:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff6317fa640 1 -- 172.21.15.16:0/2581320542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff634003990 con 0x7ff63c15e8d0 2024-09-12T01:34:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff643131640 1 -- 172.21.15.16:0/2581320542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63c09ccd0 con 0x7ff63c15e8d0 2024-09-12T01:34:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff6317fa640 1 -- 172.21.15.16:0/2581320542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634003d80 con 0x7ff63c15e8d0 2024-09-12T01:34:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.338+0000 7ff643131640 1 -- 172.21.15.16:0/2581320542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63c09d290 con 0x7ff63c15e8d0 2024-09-12T01:34:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.339+0000 7ff643131640 1 -- 172.21.15.16:0/2581320542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63c0716b0 con 0x7ff63c15e8d0 2024-09-12T01:34:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.340+0000 7ff6317fa640 1 -- 172.21.15.16:0/2581320542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff634005d10 con 0x7ff63c15e8d0 2024-09-12T01:34:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.340+0000 7ff6317fa640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff62806be30 0x7ff62806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:37.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.340+0000 7ff6317fa640 1 -- 172.21.15.16:0/2581320542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7ff63408b5c0 con 0x7ff63c15e8d0 2024-09-12T01:34:37.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.340+0000 7ff640ecd640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff62806be30 0x7ff62806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:37.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.341+0000 7ff640ecd640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff62806be30 0x7ff62806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff638000f60 tx=0x7ff638009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:37.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.344+0000 7ff6317fa640 1 -- 172.21.15.16:0/2581320542 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff63c0716b0 con 0x7ff63c15e8d0 2024-09-12T01:34:37.543 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-09-12T01:34:37.543+0000 7f404d243640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T01:34:37.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.558+0000 7ff643131640 1 -- 172.21.15.16:0/2581320542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff63c0d0d40 con 0x7ff63c15e8d0 2024-09-12T01:34:37.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.558+0000 7ff6317fa640 1 -- 172.21.15.16:0/2581320542 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5216 (secure 0 0 0) 0x7ff63c0d0d40 con 0x7ff63c15e8d0 2024-09-12T01:34:37.558 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:37.559 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"btime":"2024-09-12T01:34:34:193390+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/842803487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":842803487},{"type":"v1","addr":"172.21.15.33:6833","nonce":842803487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/70052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":70052721},{"type":"v1","addr":"172.21.15.33:6835","nonce":70052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3777111462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3777111462},{"type":"v1","addr":"172.21.15.16:6837","nonce":3777111462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:34:06.918906+0000","modified":"2024-09-12T01:34:34.193385+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.16:6835/2095784417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2095784417},{"type":"v1","addr":"172.21.15.16:6835","nonce":2095784417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4929,"qdb_cluster":[4929]},"id":2}]} 2024-09-12T01:34:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 22 2024-09-12T01:34:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.561+0000 7ff643131640 1 -- 172.21.15.16:0/2581320542 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff62806be30 msgr2=0x7ff62806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.561+0000 7ff643131640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff62806be30 0x7ff62806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff638000f60 tx=0x7ff638009040 comp rx=0 tx=0).stop 2024-09-12T01:34:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.561+0000 7ff643131640 1 -- 172.21.15.16:0/2581320542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff63c15e8d0 msgr2=0x7ff63c06e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.561+0000 7ff643131640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff63c15e8d0 0x7ff63c06e300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff63400efc0 tx=0x7ff63400c490 comp rx=0 tx=0).stop 2024-09-12T01:34:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.561+0000 7ff643131640 1 -- 172.21.15.16:0/2581320542 shutdown_connections 2024-09-12T01:34:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.561+0000 7ff643131640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff62806be30 0x7ff62806e2f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.561+0000 7ff643131640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff63c15e8d0 0x7ff63c06e300 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.561+0000 7ff643131640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c0cb580 0x7ff63c09c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.561+0000 7ff643131640 1 --2- 172.21.15.16:0/2581320542 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff63c06dd90 0x7ff63c148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.561+0000 7ff643131640 1 -- 172.21.15.16:0/2581320542 >> 172.21.15.16:0/2581320542 conn(0x7ff63c0cffa0 msgr2=0x7ff63c15ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.561+0000 7ff643131640 1 -- 172.21.15.16:0/2581320542 shutdown_connections 2024-09-12T01:34:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.561+0000 7ff643131640 1 -- 172.21.15.16:0/2581320542 wait complete. 2024-09-12T01:34:37.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:34:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.675+0000 7f83b83e4640 1 Processor -- start 2024-09-12T01:34:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.675+0000 7f83b83e4640 1 -- start start 2024-09-12T01:34:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b83e4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f83b015e3f0 0x7f83b006e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b83e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83b006cd10 0x7f83b006ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b83e4640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f83b015e7d0 0x7f83b006baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b83e4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f83b00cf430 con 0x7f83b006cd10 2024-09-12T01:34:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b83e4640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f83b00cf150 con 0x7f83b015e7d0 2024-09-12T01:34:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b83e4640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f83b00ae880 con 0x7f83b015e3f0 2024-09-12T01:34:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b597f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83b006cd10 0x7f83b006ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b597f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83b006cd10 0x7f83b006ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43248/0 (socket says 172.21.15.16:43248) 2024-09-12T01:34:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b597f640 1 -- 172.21.15.16:0/193886172 learned_addr learned my addr 172.21.15.16:0/193886172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b6180640 1 --2- 172.21.15.16:0/193886172 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f83b015e3f0 0x7f83b006e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b6981640 1 --2- 172.21.15.16:0/193886172 >> v2:172.21.15.33:3300/0 conn(0x7f83b015e7d0 0x7f83b006baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b597f640 1 -- 172.21.15.16:0/193886172 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f83b015e3f0 msgr2=0x7f83b006e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b597f640 1 --2- 172.21.15.16:0/193886172 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f83b015e3f0 0x7f83b006e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b597f640 1 -- 172.21.15.16:0/193886172 >> v2:172.21.15.33:3300/0 conn(0x7f83b015e7d0 msgr2=0x7f83b006baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b597f640 1 --2- 172.21.15.16:0/193886172 >> v2:172.21.15.33:3300/0 conn(0x7f83b015e7d0 0x7f83b006baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.676+0000 7f83b597f640 1 -- 172.21.15.16:0/193886172 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b00aeb60 con 0x7f83b006cd10 2024-09-12T01:34:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.677+0000 7f83b597f640 1 --2- 172.21.15.16:0/193886172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b006cd10 0x7f83b006ebd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f83b006d150 tx=0x7f83a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58f00c308bcb861 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.677+0000 7f83b517e640 1 -- 172.21.15.16:0/193886172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0037070 con 0x7f83b006cd10 2024-09-12T01:34:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.677+0000 7f83b517e640 1 -- 172.21.15.16:0/193886172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a0032440 con 0x7f83b006cd10 2024-09-12T01:34:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.677+0000 7f83b517e640 1 -- 172.21.15.16:0/193886172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0032810 con 0x7f83b006cd10 2024-09-12T01:34:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.678+0000 7f83b83e4640 1 -- 172.21.15.16:0/193886172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b006cd10 msgr2=0x7f83b006ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.678+0000 7f83b83e4640 1 --2- 172.21.15.16:0/193886172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b006cd10 0x7f83b006ebd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f83b006d150 tx=0x7f83a002a1e0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.678+0000 7f83b83e4640 1 -- 172.21.15.16:0/193886172 shutdown_connections 2024-09-12T01:34:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.678+0000 7f83b83e4640 1 --2- 172.21.15.16:0/193886172 >> v2:172.21.15.33:3300/0 conn(0x7f83b015e7d0 0x7f83b006baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.678+0000 7f83b83e4640 1 --2- 172.21.15.16:0/193886172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b006cd10 0x7f83b006ebd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.678+0000 7f83b83e4640 1 --2- 172.21.15.16:0/193886172 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f83b015e3f0 0x7f83b006e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.678+0000 7f83b83e4640 1 -- 172.21.15.16:0/193886172 >> 172.21.15.16:0/193886172 conn(0x7f83b00ce2d0 msgr2=0x7f83b00723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.678+0000 7f83b83e4640 1 -- 172.21.15.16:0/193886172 shutdown_connections 2024-09-12T01:34:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.678+0000 7f83b83e4640 1 -- 172.21.15.16:0/193886172 wait complete. 2024-09-12T01:34:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.679+0000 7f83b83e4640 1 Processor -- start 2024-09-12T01:34:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.679+0000 7f83b83e4640 1 -- start start 2024-09-12T01:34:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b83e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b006cd10 0x7f83b0154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b83e4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83b015e3f0 0x7f83b014a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b83e4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f83b015e7d0 0x7f83b014aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b83e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83b00d0010 con 0x7f83b006cd10 2024-09-12T01:34:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b83e4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f83b00b3980 con 0x7f83b015e3f0 2024-09-12T01:34:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b83e4640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f83b00d0190 con 0x7f83b015e7d0 2024-09-12T01:34:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b6180640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b006cd10 0x7f83b0154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b6180640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b006cd10 0x7f83b0154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43250/0 (socket says 172.21.15.16:43250) 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b6180640 1 -- 172.21.15.16:0/1899979985 learned_addr learned my addr 172.21.15.16:0/1899979985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b597f640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83b015e3f0 0x7f83b014a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b6981640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f83b015e7d0 0x7f83b014aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b6180640 1 -- 172.21.15.16:0/1899979985 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f83b015e7d0 msgr2=0x7f83b014aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b6180640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f83b015e7d0 0x7f83b014aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b6180640 1 -- 172.21.15.16:0/1899979985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83b015e3f0 msgr2=0x7f83b014a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b6180640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83b015e3f0 0x7f83b014a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.680+0000 7f83b6180640 1 -- 172.21.15.16:0/1899979985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b015adf0 con 0x7f83b006cd10 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.681+0000 7f83b6180640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b006cd10 0x7f83b0154f50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00e890 tx=0x7f83ac00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.681+0000 7f83a6ffd640 1 -- 172.21.15.16:0/1899979985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac017070 con 0x7f83b006cd10 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.681+0000 7f83a6ffd640 1 -- 172.21.15.16:0/1899979985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ac00c940 con 0x7f83b006cd10 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.681+0000 7f83b83e4640 1 -- 172.21.15.16:0/1899979985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b0157e50 con 0x7f83b006cd10 2024-09-12T01:34:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.681+0000 7f83a6ffd640 1 -- 172.21.15.16:0/1899979985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00cd10 con 0x7f83b006cd10 2024-09-12T01:34:37.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.681+0000 7f83b83e4640 1 -- 172.21.15.16:0/1899979985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b0158410 con 0x7f83b006cd10 2024-09-12T01:34:37.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.682+0000 7f83b83e4640 1 -- 172.21.15.16:0/1899979985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b00ceb60 con 0x7f83b006cd10 2024-09-12T01:34:37.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.682+0000 7f83a6ffd640 1 -- 172.21.15.16:0/1899979985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83ac01a070 con 0x7f83b006cd10 2024-09-12T01:34:37.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.683+0000 7f83a6ffd640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f839406be30 0x7f839406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:37.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.683+0000 7f83a6ffd640 1 -- 172.21.15.16:0/1899979985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f83ac08b2f0 con 0x7f83b006cd10 2024-09-12T01:34:37.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.683+0000 7f83b597f640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f839406be30 0x7f839406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:37.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.683+0000 7f83b597f640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f839406be30 0x7f839406e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f83a0034000 tx=0x7f83a002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:37.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.687+0000 7f83a6ffd640 1 -- 172.21.15.16:0/1899979985 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f83ac059460 con 0x7f83b006cd10 2024-09-12T01:34:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.897+0000 7f83b83e4640 1 -- 172.21.15.16:0/1899979985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83b00ccf90 con 0x7f83b006cd10 2024-09-12T01:34:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.898+0000 7f83a6ffd640 1 -- 172.21.15.16:0/1899979985 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5216 (secure 0 0 0) 0x7f83ac05d760 con 0x7f83b006cd10 2024-09-12T01:34:37.897 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:37.898 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"btime":"2024-09-12T01:34:34:193390+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/842803487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":842803487},{"type":"v1","addr":"172.21.15.33:6833","nonce":842803487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/70052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":70052721},{"type":"v1","addr":"172.21.15.33:6835","nonce":70052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3777111462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3777111462},{"type":"v1","addr":"172.21.15.16:6837","nonce":3777111462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:34:06.918906+0000","modified":"2024-09-12T01:34:34.193385+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.16:6835/2095784417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2095784417},{"type":"v1","addr":"172.21.15.16:6835","nonce":2095784417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4929,"qdb_cluster":[4929]},"id":2}]} 2024-09-12T01:34:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 22 2024-09-12T01:34:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.899+0000 7f83b83e4640 1 -- 172.21.15.16:0/1899979985 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f839406be30 msgr2=0x7f839406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.899+0000 7f83b83e4640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f839406be30 0x7f839406e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f83a0034000 tx=0x7f83a002e000 comp rx=0 tx=0).stop 2024-09-12T01:34:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.899+0000 7f83b83e4640 1 -- 172.21.15.16:0/1899979985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b006cd10 msgr2=0x7f83b0154f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.899+0000 7f83b83e4640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b006cd10 0x7f83b0154f50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00e890 tx=0x7f83ac00ed50 comp rx=0 tx=0).stop 2024-09-12T01:34:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.899+0000 7f83b83e4640 1 -- 172.21.15.16:0/1899979985 shutdown_connections 2024-09-12T01:34:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.899+0000 7f83b83e4640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f839406be30 0x7f839406e2f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.899+0000 7f83b83e4640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f83b015e7d0 0x7f83b014aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.899+0000 7f83b83e4640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83b015e3f0 0x7f83b014a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.899+0000 7f83b83e4640 1 --2- 172.21.15.16:0/1899979985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b006cd10 0x7f83b0154f50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.899+0000 7f83b83e4640 1 -- 172.21.15.16:0/1899979985 >> 172.21.15.16:0/1899979985 conn(0x7f83b00ce2d0 msgr2=0x7f83b00b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.900+0000 7f83b83e4640 1 -- 172.21.15.16:0/1899979985 shutdown_connections 2024-09-12T01:34:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:37.900+0000 7f83b83e4640 1 -- 172.21.15.16:0/1899979985 wait complete. 2024-09-12T01:34:37.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:34:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.013+0000 7ff8b9be4640 1 Processor -- start 2024-09-12T01:34:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.013+0000 7ff8b9be4640 1 -- start start 2024-09-12T01:34:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.013+0000 7ff8b9be4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8b4156940 0x7ff8b415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.013+0000 7ff8b9be4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8b41608e0 con 0x7ff8b40ccf30 2024-09-12T01:34:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.013+0000 7ff8b9be4640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8b4160bc0 con 0x7ff8b415ff00 2024-09-12T01:34:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.013+0000 7ff8b9be4640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b415ce90 con 0x7ff8b4156940 2024-09-12T01:34:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff8b40ccf30 0x7ff8b40cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58210/0 (socket says 172.21.15.16:58210) 2024-09-12T01:34:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b3fff640 1 -- 172.21.15.16:0/3033756393 learned_addr learned my addr 172.21.15.16:0/3033756393 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b37fe640 1 --2- 172.21.15.16:0/3033756393 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8b4156940 0x7ff8b415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2406387482 0 0) 0x7ff8b41608e0 con 0x7ff8b40ccf30 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8a0003520 con 0x7ff8b40ccf30 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4172640387 0 0) 0x7ff8b4160bc0 con 0x7ff8b415ff00 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8b41608e0 con 0x7ff8b415ff00 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3641284625 0 0) 0x7ff8a0003520 con 0x7ff8b40ccf30 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8b4156940 msgr2=0x7ff8b415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 --2- 172.21.15.16:0/3033756393 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8b4156940 0x7ff8b415d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 >> v1:172.21.15.33:6789/0 conn(0x7ff8b415ff00 legacy=0x7ff8b40cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b415cb20 con 0x7ff8b40ccf30 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7ff8ac0019f0 con 0x7ff8b40ccf30 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 >> v1:172.21.15.16:6789/0 conn(0x7ff8b40ccf30 legacy=0x7ff8b40cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 --2- 172.21.15.16:0/3033756393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0003be0 0x7ff8a0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 --2- 172.21.15.16:0/3033756393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a0004800 0x7ff8a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 --2- 172.21.15.16:0/3033756393 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8b4156940 0x7ff8a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b415ce90 con 0x7ff8a0004800 2024-09-12T01:34:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a000baa0 con 0x7ff8a0003be0 2024-09-12T01:34:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a000bca0 con 0x7ff8b4156940 2024-09-12T01:34:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b2ffd640 1 --2- 172.21.15.16:0/3033756393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a0004800 0x7ff8a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b37fe640 1 --2- 172.21.15.16:0/3033756393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0003be0 0x7ff8a0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.014+0000 7ff8b3fff640 1 --2- 172.21.15.16:0/3033756393 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8b4156940 0x7ff8a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.015+0000 7ff8b2ffd640 1 -- 172.21.15.16:0/3033756393 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8b4156940 msgr2=0x7ff8a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.015+0000 7ff8b2ffd640 1 --2- 172.21.15.16:0/3033756393 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8b4156940 0x7ff8a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.015+0000 7ff8b2ffd640 1 -- 172.21.15.16:0/3033756393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0003be0 msgr2=0x7ff8a0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.015+0000 7ff8b2ffd640 1 --2- 172.21.15.16:0/3033756393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0003be0 0x7ff8a0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.015+0000 7ff8b2ffd640 1 -- 172.21.15.16:0/3033756393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a0003520 con 0x7ff8a0004800 2024-09-12T01:34:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.015+0000 7ff8b2ffd640 1 --2- 172.21.15.16:0/3033756393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a0004800 0x7ff8a0004c80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4072900 tx=0x7ff8a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404eb474dd190fbc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.015+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0014f0 con 0x7ff8a0004800 2024-09-12T01:34:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.016+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff89c0052d0 con 0x7ff8a0004800 2024-09-12T01:34:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.016+0000 7ff8b27fc640 1 -- 172.21.15.16:0/3033756393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c02ac00 con 0x7ff8a0004800 2024-09-12T01:34:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.016+0000 7ff8b9be4640 1 -- 172.21.15.16:0/3033756393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a0004800 msgr2=0x7ff8a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.016+0000 7ff8b9be4640 1 --2- 172.21.15.16:0/3033756393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a0004800 0x7ff8a0004c80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4072900 tx=0x7ff8a0005830 comp rx=0 tx=0).stop 2024-09-12T01:34:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.016+0000 7ff8b3fff640 1 -- 172.21.15.16:0/3033756393 reap_dead start 2024-09-12T01:34:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.016+0000 7ff8b9be4640 1 -- 172.21.15.16:0/3033756393 shutdown_connections 2024-09-12T01:34:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.016+0000 7ff8b9be4640 1 -- 172.21.15.16:0/3033756393 >> 172.21.15.16:0/3033756393 conn(0x7ff8b406fdc0 msgr2=0x7ff8b40701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.016+0000 7ff8b9be4640 1 -- 172.21.15.16:0/3033756393 shutdown_connections 2024-09-12T01:34:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.016+0000 7ff8b9be4640 1 -- 172.21.15.16:0/3033756393 wait complete. 2024-09-12T01:34:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.017+0000 7ff8b9be4640 1 Processor -- start 2024-09-12T01:34:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.017+0000 7ff8b9be4640 1 -- start start 2024-09-12T01:34:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.018+0000 7ff8b9be4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a0004800 0x7ff8b4155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.018+0000 7ff8b9be4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8b40b6a80 0x7ff8b414d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.018+0000 7ff8b9be4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8b40c92a0 0x7ff8b40c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.018+0000 7ff8b9be4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b40d1b30 con 0x7ff8a0004800 2024-09-12T01:34:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.018+0000 7ff8b9be4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b40d19b0 con 0x7ff8b40c92a0 2024-09-12T01:34:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.018+0000 7ff8b9be4640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b4160360 con 0x7ff8b40b6a80 2024-09-12T01:34:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.018+0000 7ff8b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a0004800 0x7ff8b4155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.018+0000 7ff8b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a0004800 0x7ff8b4155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43280/0 (socket says 172.21.15.16:43280) 2024-09-12T01:34:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.018+0000 7ff8b37fe640 1 -- 172.21.15.16:0/2658485170 learned_addr learned my addr 172.21.15.16:0/2658485170 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.018+0000 7ff8b2ffd640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8b40b6a80 0x7ff8b414d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.018+0000 7ff8b3fff640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8b40c92a0 0x7ff8b40c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.019+0000 7ff8b37fe640 1 -- 172.21.15.16:0/2658485170 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8b40b6a80 msgr2=0x7ff8b414d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.019+0000 7ff8b37fe640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8b40b6a80 0x7ff8b414d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.019+0000 7ff8b37fe640 1 -- 172.21.15.16:0/2658485170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8b40c92a0 msgr2=0x7ff8b40c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.019+0000 7ff8b37fe640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8b40c92a0 0x7ff8b40c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.019+0000 7ff8b37fe640 1 -- 172.21.15.16:0/2658485170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b4157c80 con 0x7ff8a0004800 2024-09-12T01:34:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.019+0000 7ff8b37fe640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a0004800 0x7ff8b4155d20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4011a00 tx=0x7ff8a4011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.019+0000 7ff8abfff640 1 -- 172.21.15.16:0/2658485170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a400fb50 con 0x7ff8a0004800 2024-09-12T01:34:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.019+0000 7ff8abfff640 1 -- 172.21.15.16:0/2658485170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8a400fcf0 con 0x7ff8a0004800 2024-09-12T01:34:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.019+0000 7ff8abfff640 1 -- 172.21.15.16:0/2658485170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a400fea0 con 0x7ff8a0004800 2024-09-12T01:34:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.019+0000 7ff8b9be4640 1 -- 172.21.15.16:0/2658485170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b40b6ff0 con 0x7ff8a0004800 2024-09-12T01:34:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.019+0000 7ff8b9be4640 1 -- 172.21.15.16:0/2658485170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b40b75b0 con 0x7ff8a0004800 2024-09-12T01:34:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.020+0000 7ff8b9be4640 1 -- 172.21.15.16:0/2658485170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b40cc820 con 0x7ff8a0004800 2024-09-12T01:34:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.020+0000 7ff8abfff640 1 -- 172.21.15.16:0/2658485170 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8a4012040 con 0x7ff8a0004800 2024-09-12T01:34:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.021+0000 7ff8abfff640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff89406be30 0x7ff89406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.021+0000 7ff8abfff640 1 -- 172.21.15.16:0/2658485170 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7ff8a408b560 con 0x7ff8a0004800 2024-09-12T01:34:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.021+0000 7ff8b2ffd640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff89406be30 0x7ff89406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.021+0000 7ff8b2ffd640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff89406be30 0x7ff89406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff89c032000 tx=0x7ff89c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:38.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.025+0000 7ff8abfff640 1 -- 172.21.15.16:0/2658485170 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff8a4019030 con 0x7ff8a0004800 2024-09-12T01:34:38.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.205+0000 7ff8b9be4640 1 -- 172.21.15.16:0/2658485170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8b40d08a0 con 0x7ff8a0004800 2024-09-12T01:34:38.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.206+0000 7ff8abfff640 1 -- 172.21.15.16:0/2658485170 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17107 (secure 0 0 0) 0x7ff8a40596e0 con 0x7ff8a0004800 2024-09-12T01:34:38.207 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:38.207 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:34:30.165532+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:34:04.823039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-12T01:34:05.900693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6837/4214019228":"2024-09-13T01:34:30.165507+0000","172.21.15.16:6836/4214019228":"2024-09-13T01:34:30.165507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:34:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.208+0000 7ff8b9be4640 1 -- 172.21.15.16:0/2658485170 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff89406be30 msgr2=0x7ff89406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.208+0000 7ff8b9be4640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff89406be30 0x7ff89406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff89c032000 tx=0x7ff89c009030 comp rx=0 tx=0).stop 2024-09-12T01:34:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.209+0000 7ff8b9be4640 1 -- 172.21.15.16:0/2658485170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a0004800 msgr2=0x7ff8b4155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.209+0000 7ff8b9be4640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a0004800 0x7ff8b4155d20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4011a00 tx=0x7ff8a4011ec0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.209+0000 7ff8b9be4640 1 -- 172.21.15.16:0/2658485170 shutdown_connections 2024-09-12T01:34:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.209+0000 7ff8b9be4640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff89406be30 0x7ff89406e2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.209+0000 7ff8b9be4640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8b40c92a0 0x7ff8b40c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.209+0000 7ff8b9be4640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff8b40b6a80 0x7ff8b414d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.209+0000 7ff8b9be4640 1 --2- 172.21.15.16:0/2658485170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a0004800 0x7ff8b4155d20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.209+0000 7ff8b9be4640 1 -- 172.21.15.16:0/2658485170 >> 172.21.15.16:0/2658485170 conn(0x7ff8b406fdc0 msgr2=0x7ff8b415f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.209+0000 7ff8b9be4640 1 -- 172.21.15.16:0/2658485170 shutdown_connections 2024-09-12T01:34:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.209+0000 7ff8b9be4640 1 -- 172.21.15.16:0/2658485170 wait complete. 2024-09-12T01:34:38.220 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T01:34:38.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:34:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.323+0000 7f81b82d9640 1 Processor -- start 2024-09-12T01:34:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.323+0000 7f81b82d9640 1 -- start start 2024-09-12T01:34:38.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.324+0000 7f81b82d9640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81b0067bd0 0x7f81b00b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.324+0000 7f81b82d9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81b00b61d0 0x7f81b006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.324+0000 7f81b82d9640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b0160060 con 0x7f81b015c2c0 2024-09-12T01:34:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.324+0000 7f81b82d9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f81b00b6620 con 0x7f81b00b61d0 2024-09-12T01:34:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.324+0000 7f81b82d9640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f81b0160340 con 0x7f81b0067bd0 2024-09-12T01:34:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.324+0000 7f81b6876640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81b00b61d0 0x7f81b006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.324+0000 7f81b6876640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81b00b61d0 0x7f81b006bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43286/0 (socket says 172.21.15.16:43286) 2024-09-12T01:34:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.324+0000 7f81b6876640 1 -- 172.21.15.16:0/2939466035 learned_addr learned my addr 172.21.15.16:0/2939466035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.324+0000 7f81b5874640 1 --2- 172.21.15.16:0/2939466035 >> v2:172.21.15.33:3300/0 conn(0x7f81b0067bd0 0x7f81b00b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.324+0000 7f81b6876640 1 -- 172.21.15.16:0/2939466035 >> v2:172.21.15.33:3300/0 conn(0x7f81b0067bd0 msgr2=0x7f81b00b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.324+0000 7f81b6876640 1 --2- 172.21.15.16:0/2939466035 >> v2:172.21.15.33:3300/0 conn(0x7f81b0067bd0 0x7f81b00b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.324+0000 7f81b6876640 1 -- 172.21.15.16:0/2939466035 >> v1:172.21.15.33:6789/0 conn(0x7f81b015c2c0 legacy=0x7f81b00cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.324+0000 7f81b6876640 1 -- 172.21.15.16:0/2939466035 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b0160620 con 0x7f81b00b61d0 2024-09-12T01:34:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.325+0000 7f81b6876640 1 --2- 172.21.15.16:0/2939466035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b00b61d0 0x7f81b006bbf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f81b00b35c0 tx=0x7f81a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918f4fc3694079dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.325+0000 7f81b5073640 1 -- 172.21.15.16:0/2939466035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8036070 con 0x7f81b00b61d0 2024-09-12T01:34:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.325+0000 7f81b5073640 1 -- 172.21.15.16:0/2939466035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a8031600 con 0x7f81b00b61d0 2024-09-12T01:34:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.325+0000 7f81b5073640 1 -- 172.21.15.16:0/2939466035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a80319d0 con 0x7f81b00b61d0 2024-09-12T01:34:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.326+0000 7f81b82d9640 1 -- 172.21.15.16:0/2939466035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b00b61d0 msgr2=0x7f81b006bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.326+0000 7f81b82d9640 1 --2- 172.21.15.16:0/2939466035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b00b61d0 0x7f81b006bbf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f81b00b35c0 tx=0x7f81a802a6a0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.326+0000 7f81b82d9640 1 -- 172.21.15.16:0/2939466035 shutdown_connections 2024-09-12T01:34:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.326+0000 7f81b82d9640 1 --2- 172.21.15.16:0/2939466035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b00b61d0 0x7f81b006bbf0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.326+0000 7f81b82d9640 1 --2- 172.21.15.16:0/2939466035 >> v2:172.21.15.33:3300/0 conn(0x7f81b0067bd0 0x7f81b00b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.326+0000 7f81b82d9640 1 -- 172.21.15.16:0/2939466035 >> 172.21.15.16:0/2939466035 conn(0x7f81b00ce2d0 msgr2=0x7f81b00723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.326+0000 7f81b82d9640 1 -- 172.21.15.16:0/2939466035 shutdown_connections 2024-09-12T01:34:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.326+0000 7f81b82d9640 1 -- 172.21.15.16:0/2939466035 wait complete. 2024-09-12T01:34:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.327+0000 7f81b82d9640 1 Processor -- start 2024-09-12T01:34:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.327+0000 7f81b82d9640 1 -- start start 2024-09-12T01:34:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b82d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b0067bd0 0x7f81b0156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b82d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b00b61d0 0x7f81b00b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b82d9640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f81b015c2c0 0x7f81b01f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b82d9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81b015c6a0 con 0x7f81b0067bd0 2024-09-12T01:34:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b82d9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0160be0 con 0x7f81b00b61d0 2024-09-12T01:34:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b82d9640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f81b015c820 con 0x7f81b015c2c0 2024-09-12T01:34:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b6075640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b0067bd0 0x7f81b0156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b6075640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b0067bd0 0x7f81b0156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43296/0 (socket says 172.21.15.16:43296) 2024-09-12T01:34:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b6075640 1 -- 172.21.15.16:0/2586605538 learned_addr learned my addr 172.21.15.16:0/2586605538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b6075640 1 -- 172.21.15.16:0/2586605538 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f81b015c2c0 msgr2=0x7f81b01f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b6075640 1 --2- 172.21.15.16:0/2586605538 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f81b015c2c0 0x7f81b01f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b6075640 1 -- 172.21.15.16:0/2586605538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b00b61d0 msgr2=0x7f81b00b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b5874640 1 --2- 172.21.15.16:0/2586605538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b00b61d0 0x7f81b00b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b6075640 1 --2- 172.21.15.16:0/2586605538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b00b61d0 0x7f81b00b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.328+0000 7f81b6075640 1 -- 172.21.15.16:0/2586605538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b00b5870 con 0x7f81b0067bd0 2024-09-12T01:34:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.329+0000 7f81b6075640 1 --2- 172.21.15.16:0/2586605538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b0067bd0 0x7f81b0156ba0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00c3c0 tx=0x7f81ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.329+0000 7f81a6ffd640 1 -- 172.21.15.16:0/2586605538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0052d0 con 0x7f81b0067bd0 2024-09-12T01:34:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.329+0000 7f81a6ffd640 1 -- 172.21.15.16:0/2586605538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ac005490 con 0x7f81b0067bd0 2024-09-12T01:34:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.329+0000 7f81b82d9640 1 -- 172.21.15.16:0/2586605538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b015aa70 con 0x7f81b0067bd0 2024-09-12T01:34:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.329+0000 7f81a6ffd640 1 -- 172.21.15.16:0/2586605538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac007260 con 0x7f81b0067bd0 2024-09-12T01:34:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.329+0000 7f81b82d9640 1 -- 172.21.15.16:0/2586605538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b01f7e30 con 0x7f81b0067bd0 2024-09-12T01:34:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.330+0000 7f81b82d9640 1 -- 172.21.15.16:0/2586605538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b0068160 con 0x7f81b0067bd0 2024-09-12T01:34:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.330+0000 7f81a6ffd640 1 -- 172.21.15.16:0/2586605538 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81ac007400 con 0x7f81b0067bd0 2024-09-12T01:34:38.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.331+0000 7f81a6ffd640 1 --2- 172.21.15.16:0/2586605538 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f819406be30 0x7f819406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.331+0000 7f81a6ffd640 1 -- 172.21.15.16:0/2586605538 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f81ac08a460 con 0x7f81b0067bd0 2024-09-12T01:34:38.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.331+0000 7f81b5874640 1 --2- 172.21.15.16:0/2586605538 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f819406be30 0x7f819406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.331+0000 7f81b5874640 1 --2- 172.21.15.16:0/2586605538 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f819406be30 0x7f819406e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f81b01f5880 tx=0x7f81a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.335+0000 7f81a6ffd640 1 -- 172.21.15.16:0/2586605538 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f81ac058cc0 con 0x7f81b0067bd0 2024-09-12T01:34:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.548+0000 7f81b82d9640 1 -- 172.21.15.16:0/2586605538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81b006c390 con 0x7f81b0067bd0 2024-09-12T01:34:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.548+0000 7f81a6ffd640 1 -- 172.21.15.16:0/2586605538 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5216 (secure 0 0 0) 0x7f81ac0585c0 con 0x7f81b0067bd0 2024-09-12T01:34:38.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:38.548 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"btime":"2024-09-12T01:34:34:193390+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/842803487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":842803487},{"type":"v1","addr":"172.21.15.33:6833","nonce":842803487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/70052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":70052721},{"type":"v1","addr":"172.21.15.33:6835","nonce":70052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3777111462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3777111462},{"type":"v1","addr":"172.21.15.16:6837","nonce":3777111462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:34:06.918906+0000","modified":"2024-09-12T01:34:34.193385+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.16:6835/2095784417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2095784417},{"type":"v1","addr":"172.21.15.16:6835","nonce":2095784417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4929,"qdb_cluster":[4929]},"id":2}]} 2024-09-12T01:34:38.549 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 22 2024-09-12T01:34:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.550+0000 7f81b82d9640 1 -- 172.21.15.16:0/2586605538 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f819406be30 msgr2=0x7f819406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.550+0000 7f81b82d9640 1 --2- 172.21.15.16:0/2586605538 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f819406be30 0x7f819406e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f81b01f5880 tx=0x7f81a0008040 comp rx=0 tx=0).stop 2024-09-12T01:34:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.550+0000 7f81b82d9640 1 -- 172.21.15.16:0/2586605538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b0067bd0 msgr2=0x7f81b0156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.550+0000 7f81b82d9640 1 --2- 172.21.15.16:0/2586605538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b0067bd0 0x7f81b0156ba0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00c3c0 tx=0x7f81ac00c880 comp rx=0 tx=0).stop 2024-09-12T01:34:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.550+0000 7f81b82d9640 1 -- 172.21.15.16:0/2586605538 shutdown_connections 2024-09-12T01:34:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.550+0000 7f81b82d9640 1 --2- 172.21.15.16:0/2586605538 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f819406be30 0x7f819406e2f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.550+0000 7f81b82d9640 1 --2- 172.21.15.16:0/2586605538 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f81b015c2c0 0x7f81b01f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.550+0000 7f81b82d9640 1 --2- 172.21.15.16:0/2586605538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b00b61d0 0x7f81b00b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.550+0000 7f81b82d9640 1 --2- 172.21.15.16:0/2586605538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b0067bd0 0x7f81b0156ba0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.550+0000 7f81b82d9640 1 -- 172.21.15.16:0/2586605538 >> 172.21.15.16:0/2586605538 conn(0x7f81b00ce2d0 msgr2=0x7f81b00ccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.550+0000 7f81b82d9640 1 -- 172.21.15.16:0/2586605538 shutdown_connections 2024-09-12T01:34:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.550+0000 7f81b82d9640 1 -- 172.21.15.16:0/2586605538 wait complete. 2024-09-12T01:34:38.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:34:38.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.664+0000 7fa7ae7ed640 1 Processor -- start 2024-09-12T01:34:38.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.664+0000 7fa7ae7ed640 1 -- start start 2024-09-12T01:34:38.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7ae7ed640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa7a815e800 0x7fa7a80cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7ae7ed640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa7a80cf910 0x7fa7a806d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7ae7ed640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7a806dbd0 0x7fa7a806e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7ae7ed640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa7a80cfd30 con 0x7fa7a80cf910 2024-09-12T01:34:38.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7ae7ed640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fa7a815ebe0 con 0x7fa7a815e800 2024-09-12T01:34:38.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7ae7ed640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a80d42d0 con 0x7fa7a806dbd0 2024-09-12T01:34:38.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7a77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa7a80cf910 0x7fa7a806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7a77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa7a80cf910 0x7fa7a806d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43304/0 (socket says 172.21.15.16:43304) 2024-09-12T01:34:38.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7a77fe640 1 -- 172.21.15.16:0/4136157897 learned_addr learned my addr 172.21.15.16:0/4136157897 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:38.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7a7fff640 1 --2- 172.21.15.16:0/4136157897 >> v2:172.21.15.33:3300/0 conn(0x7fa7a815e800 0x7fa7a80cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7acd8a640 1 --2- 172.21.15.16:0/4136157897 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7a806dbd0 0x7fa7a806e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7a77fe640 1 -- 172.21.15.16:0/4136157897 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7a806dbd0 msgr2=0x7fa7a806e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:38.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7a77fe640 1 --2- 172.21.15.16:0/4136157897 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7a806dbd0 0x7fa7a806e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7a77fe640 1 -- 172.21.15.16:0/4136157897 >> v2:172.21.15.33:3300/0 conn(0x7fa7a815e800 msgr2=0x7fa7a80cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:38.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7a77fe640 1 --2- 172.21.15.16:0/4136157897 >> v2:172.21.15.33:3300/0 conn(0x7fa7a815e800 0x7fa7a80cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.665+0000 7fa7a77fe640 1 -- 172.21.15.16:0/4136157897 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a80d3ff0 con 0x7fa7a80cf910 2024-09-12T01:34:38.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.666+0000 7fa7a77fe640 1 --2- 172.21.15.16:0/4136157897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a80cf910 0x7fa7a806d690 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa7a806c800 tx=0x7fa79c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e995a2d5191526 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:38.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.666+0000 7fa7a6ffd640 1 -- 172.21.15.16:0/4136157897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c037070 con 0x7fa7a80cf910 2024-09-12T01:34:38.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.666+0000 7fa7a6ffd640 1 -- 172.21.15.16:0/4136157897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa79c032440 con 0x7fa7a80cf910 2024-09-12T01:34:38.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.666+0000 7fa7a6ffd640 1 -- 172.21.15.16:0/4136157897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c032810 con 0x7fa7a80cf910 2024-09-12T01:34:38.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.666+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/4136157897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a80cf910 msgr2=0x7fa7a806d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.666+0000 7fa7ae7ed640 1 --2- 172.21.15.16:0/4136157897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a80cf910 0x7fa7a806d690 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa7a806c800 tx=0x7fa79c02a1e0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.667+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/4136157897 shutdown_connections 2024-09-12T01:34:38.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.667+0000 7fa7ae7ed640 1 --2- 172.21.15.16:0/4136157897 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7a806dbd0 0x7fa7a806e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.667+0000 7fa7ae7ed640 1 --2- 172.21.15.16:0/4136157897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a80cf910 0x7fa7a806d690 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.667+0000 7fa7ae7ed640 1 --2- 172.21.15.16:0/4136157897 >> v2:172.21.15.33:3300/0 conn(0x7fa7a815e800 0x7fa7a80cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.667+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/4136157897 >> 172.21.15.16:0/4136157897 conn(0x7fa7a809d750 msgr2=0x7fa7a80d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:38.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.667+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/4136157897 shutdown_connections 2024-09-12T01:34:38.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.667+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/4136157897 wait complete. 2024-09-12T01:34:38.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.668+0000 7fa7ae7ed640 1 Processor -- start 2024-09-12T01:34:38.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.668+0000 7fa7ae7ed640 1 -- start start 2024-09-12T01:34:38.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7ae7ed640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7a806dbd0 0x7fa7a8151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7ae7ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a80cf910 0x7fa7a80c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7ae7ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a815e800 0x7fa7a80c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7ae7ed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a806d170 con 0x7fa7a80cf910 2024-09-12T01:34:38.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7ae7ed640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a806ce90 con 0x7fa7a815e800 2024-09-12T01:34:38.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7ae7ed640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a80d4e30 con 0x7fa7a806dbd0 2024-09-12T01:34:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a80cf910 0x7fa7a80c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a80cf910 0x7fa7a80c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43306/0 (socket says 172.21.15.16:43306) 2024-09-12T01:34:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7a77fe640 1 -- 172.21.15.16:0/167796277 learned_addr learned my addr 172.21.15.16:0/167796277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7a77fe640 1 -- 172.21.15.16:0/167796277 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7a806dbd0 msgr2=0x7fa7a8151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7a7fff640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7a806dbd0 0x7fa7a8151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7acd8a640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a815e800 0x7fa7a80c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7a77fe640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7a806dbd0 0x7fa7a8151300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7a77fe640 1 -- 172.21.15.16:0/167796277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a815e800 msgr2=0x7fa7a80c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7a77fe640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a815e800 0x7fa7a80c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7a77fe640 1 -- 172.21.15.16:0/167796277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a80cab30 con 0x7fa7a80cf910 2024-09-12T01:34:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.669+0000 7fa7a77fe640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a80cf910 0x7fa7a80c7e60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0097c0 tx=0x7fa79c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.670+0000 7fa7a4ff9640 1 -- 172.21.15.16:0/167796277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c041020 con 0x7fa7a80cf910 2024-09-12T01:34:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.670+0000 7fa7a4ff9640 1 -- 172.21.15.16:0/167796277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa79c03d620 con 0x7fa7a80cf910 2024-09-12T01:34:38.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.670+0000 7fa7a4ff9640 1 -- 172.21.15.16:0/167796277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c037040 con 0x7fa7a80cf910 2024-09-12T01:34:38.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.670+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/167796277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a80cae10 con 0x7fa7a80cf910 2024-09-12T01:34:38.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.670+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/167796277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a80b88e0 con 0x7fa7a80cf910 2024-09-12T01:34:38.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.671+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/167796277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a806d690 con 0x7fa7a80cf910 2024-09-12T01:34:38.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.671+0000 7fa7a4ff9640 1 -- 172.21.15.16:0/167796277 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa79c044020 con 0x7fa7a80cf910 2024-09-12T01:34:38.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.672+0000 7fa7a4ff9640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa79406be30 0x7fa79406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:38.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.672+0000 7fa7a4ff9640 1 -- 172.21.15.16:0/167796277 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7fa79c0ab6b0 con 0x7fa7a80cf910 2024-09-12T01:34:38.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.672+0000 7fa7a7fff640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa79406be30 0x7fa79406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:38.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.672+0000 7fa7a7fff640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa79406be30 0x7fa79406e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa798009a10 tx=0x7fa798008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.676+0000 7fa7a4ff9640 1 -- 172.21.15.16:0/167796277 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa79c04b030 con 0x7fa7a80cf910 2024-09-12T01:34:38.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.887+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/167796277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7a80d32f0 con 0x7fa7a80cf910 2024-09-12T01:34:38.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.888+0000 7fa7a4ff9640 1 -- 172.21.15.16:0/167796277 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5216 (secure 0 0 0) 0x7fa79c079e70 con 0x7fa7a80cf910 2024-09-12T01:34:38.887 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:38.888 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"btime":"2024-09-12T01:34:34:193390+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/842803487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":842803487},{"type":"v1","addr":"172.21.15.33:6833","nonce":842803487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/70052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":70052721},{"type":"v1","addr":"172.21.15.33:6835","nonce":70052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3777111462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3777111462},{"type":"v1","addr":"172.21.15.16:6837","nonce":3777111462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:34:06.918906+0000","modified":"2024-09-12T01:34:34.193385+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.16:6835/2095784417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2095784417},{"type":"v1","addr":"172.21.15.16:6835","nonce":2095784417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4929,"qdb_cluster":[4929]},"id":2}]} 2024-09-12T01:34:38.888 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 22 2024-09-12T01:34:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.890+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/167796277 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa79406be30 msgr2=0x7fa79406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.890+0000 7fa7ae7ed640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa79406be30 0x7fa79406e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa798009a10 tx=0x7fa798008040 comp rx=0 tx=0).stop 2024-09-12T01:34:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.890+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/167796277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a80cf910 msgr2=0x7fa7a80c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.890+0000 7fa7ae7ed640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a80cf910 0x7fa7a80c7e60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0097c0 tx=0x7fa79c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.890+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/167796277 shutdown_connections 2024-09-12T01:34:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.890+0000 7fa7ae7ed640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa79406be30 0x7fa79406e2f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.890+0000 7fa7ae7ed640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a815e800 0x7fa7a80c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.890+0000 7fa7ae7ed640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a80cf910 0x7fa7a80c7e60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.890+0000 7fa7ae7ed640 1 --2- 172.21.15.16:0/167796277 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa7a806dbd0 0x7fa7a8151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.890+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/167796277 >> 172.21.15.16:0/167796277 conn(0x7fa7a809d750 msgr2=0x7fa7a80b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.890+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/167796277 shutdown_connections 2024-09-12T01:34:38.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:38.890+0000 7fa7ae7ed640 1 -- 172.21.15.16:0/167796277 wait complete. 2024-09-12T01:34:38.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:34:39.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.003+0000 7f185ed9d640 1 Processor -- start 2024-09-12T01:34:39.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.003+0000 7f185ed9d640 1 -- start start 2024-09-12T01:34:39.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185ed9d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f185815e350 0x7f18580cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185ed9d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f18580d4920 0x7f18580b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185ed9d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18580cf600 con 0x7f18580d2dc0 2024-09-12T01:34:39.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185ed9d640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f18580cf8e0 con 0x7f18580d4920 2024-09-12T01:34:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185ed9d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f18580cfbc0 con 0x7f185815e350 2024-09-12T01:34:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185d59a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f18580d2dc0 0x7f18580d4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58222/0 (socket says 172.21.15.16:58222) 2024-09-12T01:34:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185d59a640 1 -- 172.21.15.16:0/73035689 learned_addr learned my addr 172.21.15.16:0/73035689 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185dd9b640 1 --2- 172.21.15.16:0/73035689 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f185815e350 0x7f18580cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185e59c640 1 --2- 172.21.15.16:0/73035689 >> v2:172.21.15.33:3300/0 conn(0x7f18580d4920 0x7f18580b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 628405096 0 0) 0x7f18580cf600 con 0x7f18580d2dc0 2024-09-12T01:34:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1844003520 con 0x7f18580d2dc0 2024-09-12T01:34:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3492676252 0 0) 0x7f1844003520 con 0x7f18580d2dc0 2024-09-12T01:34:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f185815e350 msgr2=0x7f18580cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185cd99640 1 --2- 172.21.15.16:0/73035689 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f185815e350 0x7f18580cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 >> v2:172.21.15.33:3300/0 conn(0x7f18580d4920 msgr2=0x7f18580b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185cd99640 1 --2- 172.21.15.16:0/73035689 >> v2:172.21.15.33:3300/0 conn(0x7f18580d4920 0x7f18580b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18580cfea0 con 0x7f18580d2dc0 2024-09-12T01:34:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f1848001670 con 0x7f18580d2dc0 2024-09-12T01:34:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.004+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 >> v1:172.21.15.16:6789/0 conn(0x7f18580d2dc0 legacy=0x7f18580d4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185cd99640 1 --2- 172.21.15.16:0/73035689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844003a20 0x7f1844003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185cd99640 1 --2- 172.21.15.16:0/73035689 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f185815e350 0x7f1844004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185cd99640 1 --2- 172.21.15.16:0/73035689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1844004d40 0x7f18440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18580cf8e0 con 0x7f1844004d40 2024-09-12T01:34:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18580cfbc0 con 0x7f1844003a20 2024-09-12T01:34:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f184400ba40 con 0x7f185815e350 2024-09-12T01:34:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185e59c640 1 --2- 172.21.15.16:0/73035689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1844004d40 0x7f18440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185dd9b640 1 --2- 172.21.15.16:0/73035689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844003a20 0x7f1844003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185d59a640 1 --2- 172.21.15.16:0/73035689 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f185815e350 0x7f1844004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185e59c640 1 -- 172.21.15.16:0/73035689 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f185815e350 msgr2=0x7f1844004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185e59c640 1 --2- 172.21.15.16:0/73035689 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f185815e350 0x7f1844004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185e59c640 1 -- 172.21.15.16:0/73035689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844003a20 msgr2=0x7f1844003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185e59c640 1 --2- 172.21.15.16:0/73035689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844003a20 0x7f1844003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.005+0000 7f185e59c640 1 -- 172.21.15.16:0/73035689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1844003520 con 0x7f1844004d40 2024-09-12T01:34:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.006+0000 7f185e59c640 1 --2- 172.21.15.16:0/73035689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1844004d40 0x7f18440051c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1850009f90 tx=0x7f1844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8e88cbbd345c46 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.006+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850009c10 con 0x7f1844004d40 2024-09-12T01:34:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.006+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1850036070 con 0x7f1844004d40 2024-09-12T01:34:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.006+0000 7f185cd99640 1 -- 172.21.15.16:0/73035689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850031840 con 0x7f1844004d40 2024-09-12T01:34:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.006+0000 7f185ed9d640 1 -- 172.21.15.16:0/73035689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1844004d40 msgr2=0x7f18440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.006+0000 7f185ed9d640 1 --2- 172.21.15.16:0/73035689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1844004d40 0x7f18440051c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1850009f90 tx=0x7f1844005830 comp rx=0 tx=0).stop 2024-09-12T01:34:39.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.006+0000 7f185e59c640 1 -- 172.21.15.16:0/73035689 reap_dead start 2024-09-12T01:34:39.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.007+0000 7f185ed9d640 1 -- 172.21.15.16:0/73035689 shutdown_connections 2024-09-12T01:34:39.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.007+0000 7f185ed9d640 1 -- 172.21.15.16:0/73035689 >> 172.21.15.16:0/73035689 conn(0x7f185808dbe0 msgr2=0x7f185809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:39.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.007+0000 7f185ed9d640 1 -- 172.21.15.16:0/73035689 shutdown_connections 2024-09-12T01:34:39.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.007+0000 7f185ed9d640 1 -- 172.21.15.16:0/73035689 wait complete. 2024-09-12T01:34:39.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.008+0000 7f185ed9d640 1 Processor -- start 2024-09-12T01:34:39.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.008+0000 7f185ed9d640 1 -- start start 2024-09-12T01:34:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844004d40 0x7f18580ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1858161580 0x7f1858154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185ed9d640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1858161a00 0x7f1858154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185ed9d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f185815e8b0 con 0x7f1858161580 2024-09-12T01:34:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185ed9d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f185815e730 con 0x7f1844004d40 2024-09-12T01:34:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185ed9d640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f18580d31e0 con 0x7f1858161a00 2024-09-12T01:34:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185d59a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1858161580 0x7f1858154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185d59a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1858161580 0x7f1858154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43330/0 (socket says 172.21.15.16:43330) 2024-09-12T01:34:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185d59a640 1 -- 172.21.15.16:0/14866399 learned_addr learned my addr 172.21.15.16:0/14866399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185dd9b640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844004d40 0x7f18580ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185e59c640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1858161a00 0x7f1858154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185d59a640 1 -- 172.21.15.16:0/14866399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1858161a00 msgr2=0x7f1858154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185d59a640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1858161a00 0x7f1858154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185d59a640 1 -- 172.21.15.16:0/14866399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844004d40 msgr2=0x7f18580ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185d59a640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844004d40 0x7f18580ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185d59a640 1 -- 172.21.15.16:0/14866399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18580cb680 con 0x7f1858161580 2024-09-12T01:34:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f185d59a640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1858161580 0x7f1858154310 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1848007340 tx=0x7f1848007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f184e7fc640 1 -- 172.21.15.16:0/14866399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800f940 con 0x7f1858161580 2024-09-12T01:34:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f184e7fc640 1 -- 172.21.15.16:0/14866399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1848018070 con 0x7f1858161580 2024-09-12T01:34:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.009+0000 7f184e7fc640 1 -- 172.21.15.16:0/14866399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800fd60 con 0x7f1858161580 2024-09-12T01:34:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.010+0000 7f185ed9d640 1 -- 172.21.15.16:0/14866399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18580ce0c0 con 0x7f1858161580 2024-09-12T01:34:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.010+0000 7f185ed9d640 1 -- 172.21.15.16:0/14866399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18580ce680 con 0x7f1858161580 2024-09-12T01:34:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.010+0000 7f185ed9d640 1 -- 172.21.15.16:0/14866399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18580cefd0 con 0x7f1858161580 2024-09-12T01:34:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.011+0000 7f184e7fc640 1 -- 172.21.15.16:0/14866399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f184801d460 con 0x7f1858161580 2024-09-12T01:34:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.011+0000 7f184e7fc640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f183c06be60 0x7f183c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.012+0000 7f184e7fc640 1 -- 172.21.15.16:0/14866399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f184808af70 con 0x7f1858161580 2024-09-12T01:34:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.012+0000 7f185dd9b640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f183c06be60 0x7f183c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.012+0000 7f185dd9b640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f183c06be60 0x7f183c06e320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f18580cb0e0 tx=0x7f1854009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.016+0000 7f184e7fc640 1 -- 172.21.15.16:0/14866399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f18580cb680 con 0x7f1858161580 2024-09-12T01:34:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.191+0000 7f185ed9d640 1 -- 172.21.15.16:0/14866399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18580b5b50 con 0x7f1858161580 2024-09-12T01:34:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.192+0000 7f184e7fc640 1 -- 172.21.15.16:0/14866399 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17107 (secure 0 0 0) 0x7f1848059730 con 0x7f1858161580 2024-09-12T01:34:39.192 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:39.192 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:34:30.165532+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:34:04.823039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-12T01:34:05.900693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6837/4214019228":"2024-09-13T01:34:30.165507+0000","172.21.15.16:6836/4214019228":"2024-09-13T01:34:30.165507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:34:39.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.194+0000 7f185ed9d640 1 -- 172.21.15.16:0/14866399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f183c06be60 msgr2=0x7f183c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.194+0000 7f185ed9d640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f183c06be60 0x7f183c06e320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f18580cb0e0 tx=0x7f1854009190 comp rx=0 tx=0).stop 2024-09-12T01:34:39.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.194+0000 7f185ed9d640 1 -- 172.21.15.16:0/14866399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1858161580 msgr2=0x7f1858154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.194+0000 7f185ed9d640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1858161580 0x7f1858154310 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1848007340 tx=0x7f1848007740 comp rx=0 tx=0).stop 2024-09-12T01:34:39.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.194+0000 7f185ed9d640 1 -- 172.21.15.16:0/14866399 shutdown_connections 2024-09-12T01:34:39.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.194+0000 7f185ed9d640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f183c06be60 0x7f183c06e320 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.194+0000 7f185ed9d640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1858161a00 0x7f1858154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.194+0000 7f185ed9d640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1858161580 0x7f1858154310 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.194+0000 7f185ed9d640 1 --2- 172.21.15.16:0/14866399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1844004d40 0x7f18580ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.194+0000 7f185ed9d640 1 -- 172.21.15.16:0/14866399 >> 172.21.15.16:0/14866399 conn(0x7f185808dbe0 msgr2=0x7f18580d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:39.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.194+0000 7f185ed9d640 1 -- 172.21.15.16:0/14866399 shutdown_connections 2024-09-12T01:34:39.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.194+0000 7f185ed9d640 1 -- 172.21.15.16:0/14866399 wait complete. 2024-09-12T01:34:39.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T01:34:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.309+0000 7ff72c903640 1 Processor -- start 2024-09-12T01:34:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.309+0000 7ff72c903640 1 -- start start 2024-09-12T01:34:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.310+0000 7ff72c903640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff72815e660 0x7ff7280d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.310+0000 7ff72c903640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7280d4d60 0x7ff7280d4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.310+0000 7ff72c903640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff728162500 con 0x7ff728162020 2024-09-12T01:34:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.310+0000 7ff72c903640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff7280d2020 con 0x7ff7280d4d60 2024-09-12T01:34:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.310+0000 7ff72c903640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff7281627e0 con 0x7ff72815e660 2024-09-12T01:34:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.310+0000 7ff725d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7280d4d60 0x7ff7280d4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.310+0000 7ff726d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff728162020 0x7ff7280b45c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58234/0 (socket says 172.21.15.16:58234) 2024-09-12T01:34:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.310+0000 7ff726d9d640 1 -- 172.21.15.16:0/1429100455 learned_addr learned my addr 172.21.15.16:0/1429100455 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.311+0000 7ff72659c640 1 --2- 172.21.15.16:0/1429100455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff72815e660 0x7ff7280d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.311+0000 7ff725d9b640 1 -- 172.21.15.16:0/1429100455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff72815e660 msgr2=0x7ff7280d4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.311+0000 7ff725d9b640 1 --2- 172.21.15.16:0/1429100455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff72815e660 0x7ff7280d4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.311+0000 7ff725d9b640 1 -- 172.21.15.16:0/1429100455 >> v1:172.21.15.16:6789/0 conn(0x7ff728162020 legacy=0x7ff7280b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.311+0000 7ff725d9b640 1 -- 172.21.15.16:0/1429100455 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff728162ac0 con 0x7ff7280d4d60 2024-09-12T01:34:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.311+0000 7ff72559a640 1 -- 172.21.15.16:0/1429100455 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1330810631 0 0) 0x7ff728162500 con 0x7ff728162020 2024-09-12T01:34:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.311+0000 7ff72659c640 1 --2- 172.21.15.16:0/1429100455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff72815e660 0x7ff7280d4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.312+0000 7ff725d9b640 1 --2- 172.21.15.16:0/1429100455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280d4d60 0x7ff7280d4640 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff7100097a0 tx=0x7ff71002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0101014dce2cafb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.312+0000 7ff72559a640 1 -- 172.21.15.16:0/1429100455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710037070 con 0x7ff7280d4d60 2024-09-12T01:34:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.312+0000 7ff72559a640 1 -- 172.21.15.16:0/1429100455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff710032440 con 0x7ff7280d4d60 2024-09-12T01:34:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.312+0000 7ff72559a640 1 -- 172.21.15.16:0/1429100455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71002ae60 con 0x7ff7280d4d60 2024-09-12T01:34:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.312+0000 7ff72c903640 1 -- 172.21.15.16:0/1429100455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280d4d60 msgr2=0x7ff7280d4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.312+0000 7ff72c903640 1 --2- 172.21.15.16:0/1429100455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280d4d60 0x7ff7280d4640 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff7100097a0 tx=0x7ff71002a110 comp rx=0 tx=0).stop 2024-09-12T01:34:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.312+0000 7ff72c903640 1 -- 172.21.15.16:0/1429100455 shutdown_connections 2024-09-12T01:34:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.312+0000 7ff72c903640 1 --2- 172.21.15.16:0/1429100455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280d4d60 0x7ff7280d4640 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.312+0000 7ff72c903640 1 --2- 172.21.15.16:0/1429100455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff72815e660 0x7ff7280d4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.312+0000 7ff72c903640 1 -- 172.21.15.16:0/1429100455 >> 172.21.15.16:0/1429100455 conn(0x7ff72808de10 msgr2=0x7ff72809d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.312+0000 7ff72c903640 1 -- 172.21.15.16:0/1429100455 shutdown_connections 2024-09-12T01:34:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.312+0000 7ff72c903640 1 -- 172.21.15.16:0/1429100455 wait complete. 2024-09-12T01:34:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.314+0000 7ff72c903640 1 Processor -- start 2024-09-12T01:34:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.314+0000 7ff72c903640 1 -- start start 2024-09-12T01:34:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.314+0000 7ff72c903640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280d4d60 0x7ff72815fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.314+0000 7ff72c903640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff72815e660 0x7ff72815f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72c903640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff728162020 0x7ff728153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72c903640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7280d0fb0 con 0x7ff7280d4d60 2024-09-12T01:34:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72c903640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7280d0e30 con 0x7ff72815e660 2024-09-12T01:34:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72c903640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7ff7280d1130 con 0x7ff728162020 2024-09-12T01:34:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280d4d60 0x7ff72815fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280d4d60 0x7ff72815fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43338/0 (socket says 172.21.15.16:43338) 2024-09-12T01:34:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72659c640 1 -- 172.21.15.16:0/3087405115 learned_addr learned my addr 172.21.15.16:0/3087405115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff725d9b640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff72815e660 0x7ff72815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff726d9d640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff728162020 0x7ff728153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72659c640 1 -- 172.21.15.16:0/3087405115 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff728162020 msgr2=0x7ff728153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72659c640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff728162020 0x7ff728153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72659c640 1 -- 172.21.15.16:0/3087405115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff72815e660 msgr2=0x7ff72815f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72659c640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff72815e660 0x7ff72815f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72659c640 1 -- 172.21.15.16:0/3087405115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7281537a0 con 0x7ff7280d4d60 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff726d9d640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff728162020 0x7ff728153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72659c640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280d4d60 0x7ff72815fd20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff72000cf60 tx=0x7ff72000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff71effd640 1 -- 172.21.15.16:0/3087405115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72001a070 con 0x7ff7280d4d60 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff71effd640 1 -- 172.21.15.16:0/3087405115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff72000d840 con 0x7ff7280d4d60 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff71effd640 1 -- 172.21.15.16:0/3087405115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72000dc10 con 0x7ff7280d4d60 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72c903640 1 -- 172.21.15.16:0/3087405115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff728160420 con 0x7ff7280d4d60 2024-09-12T01:34:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.315+0000 7ff72c903640 1 -- 172.21.15.16:0/3087405115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7280ccee0 con 0x7ff7280d4d60 2024-09-12T01:34:39.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.316+0000 7ff72c903640 1 -- 172.21.15.16:0/3087405115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7280d00e0 con 0x7ff7280d4d60 2024-09-12T01:34:39.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.317+0000 7ff71effd640 1 -- 172.21.15.16:0/3087405115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff72001d070 con 0x7ff7280d4d60 2024-09-12T01:34:39.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.317+0000 7ff71effd640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff70806be30 0x7ff70806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.317+0000 7ff71effd640 1 -- 172.21.15.16:0/3087405115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7ff72008b820 con 0x7ff7280d4d60 2024-09-12T01:34:39.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.317+0000 7ff725d9b640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff70806be30 0x7ff70806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.318+0000 7ff725d9b640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff70806be30 0x7ff70806e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff710034000 tx=0x7ff71002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.321+0000 7ff71effd640 1 -- 172.21.15.16:0/3087405115 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff7200599a0 con 0x7ff7280d4d60 2024-09-12T01:34:39.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.532+0000 7ff72c903640 1 -- 172.21.15.16:0/3087405115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff7280b4090 con 0x7ff7280d4d60 2024-09-12T01:34:39.532 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-12T01:34:39.532+0000 7fe0e42bd640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T01:34:39.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.534+0000 7ff71effd640 1 -- 172.21.15.16:0/3087405115 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) ==== 161+0+0 (secure 0 0 0) 0x7ff72005dca0 con 0x7ff7280d4d60 2024-09-12T01:34:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T01:34:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.536+0000 7ff72c903640 1 -- 172.21.15.16:0/3087405115 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff70806be30 msgr2=0x7ff70806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.537+0000 7ff72c903640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff70806be30 0x7ff70806e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff710034000 tx=0x7ff71002e000 comp rx=0 tx=0).stop 2024-09-12T01:34:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.537+0000 7ff72c903640 1 -- 172.21.15.16:0/3087405115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280d4d60 msgr2=0x7ff72815fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.537+0000 7ff72c903640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280d4d60 0x7ff72815fd20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff72000cf60 tx=0x7ff72000cf90 comp rx=0 tx=0).stop 2024-09-12T01:34:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.537+0000 7ff72c903640 1 -- 172.21.15.16:0/3087405115 shutdown_connections 2024-09-12T01:34:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.537+0000 7ff72c903640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7ff70806be30 0x7ff70806e2f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.537+0000 7ff72c903640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7ff728162020 0x7ff728153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.537+0000 7ff72c903640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff72815e660 0x7ff72815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.537+0000 7ff72c903640 1 --2- 172.21.15.16:0/3087405115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280d4d60 0x7ff72815fd20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.537+0000 7ff72c903640 1 -- 172.21.15.16:0/3087405115 >> 172.21.15.16:0/3087405115 conn(0x7ff72808de10 msgr2=0x7ff7280d24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.537+0000 7ff72c903640 1 -- 172.21.15.16:0/3087405115 shutdown_connections 2024-09-12T01:34:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.537+0000 7ff72c903640 1 -- 172.21.15.16:0/3087405115 wait complete. 2024-09-12T01:34:39.539 INFO:tasks.ceph.mds.a.smithi016.stderr: -68> 2024-09-12T01:34:37.543+0000 7f404d243640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T01:34:39.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T01:34:39.565 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-12T01:34:39.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.657+0000 7f1948cba640 1 Processor -- start 2024-09-12T01:34:39.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.657+0000 7f1948cba640 1 -- start start 2024-09-12T01:34:39.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f1948cba640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f194415e390 0x7f1944161f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f1948cba640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f19440d15f0 0x7f19440cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f1948cba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19441627e0 con 0x7f19440b75c0 2024-09-12T01:34:39.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f1948cba640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f1944162ac0 con 0x7f19440d15f0 2024-09-12T01:34:39.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f1948cba640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f1944162da0 con 0x7f194415e390 2024-09-12T01:34:39.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f1941d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f19440b75c0 0x7f19440d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58242/0 (socket says 172.21.15.16:58242) 2024-09-12T01:34:39.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f1941d9b640 1 -- 172.21.15.16:0/659375986 learned_addr learned my addr 172.21.15.16:0/659375986 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:39.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f1942d9d640 1 --2- 172.21.15.16:0/659375986 >> v2:172.21.15.33:3300/0 conn(0x7f19440d15f0 0x7f19440cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f194259c640 1 --2- 172.21.15.16:0/659375986 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f194415e390 0x7f1944161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 871088074 0 0) 0x7f19441627e0 con 0x7f19440b75c0 2024-09-12T01:34:39.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1930003520 con 0x7f19440b75c0 2024-09-12T01:34:39.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2533101548 0 0) 0x7f1930003520 con 0x7f19440b75c0 2024-09-12T01:34:39.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f194415e390 msgr2=0x7f1944161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:39.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f194159a640 1 --2- 172.21.15.16:0/659375986 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f194415e390 0x7f1944161f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 >> v2:172.21.15.33:3300/0 conn(0x7f19440d15f0 msgr2=0x7f19440cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:39.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f194159a640 1 --2- 172.21.15.16:0/659375986 >> v2:172.21.15.33:3300/0 conn(0x7f19440d15f0 0x7f19440cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19440cf4a0 con 0x7f19440b75c0 2024-09-12T01:34:39.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.658+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f192c001690 con 0x7f19440b75c0 2024-09-12T01:34:39.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 >> v1:172.21.15.16:6789/0 conn(0x7f19440b75c0 legacy=0x7f19440d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f194159a640 1 --2- 172.21.15.16:0/659375986 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f194415e390 0x7f19300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f194159a640 1 --2- 172.21.15.16:0/659375986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1930003f30 0x7f1930004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f194159a640 1 --2- 172.21.15.16:0/659375986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1930004d40 0x7f19300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1944162ac0 con 0x7f1930003f30 2024-09-12T01:34:39.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1944162da0 con 0x7f1930004d40 2024-09-12T01:34:39.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f193000b9f0 con 0x7f194415e390 2024-09-12T01:34:39.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f1941d9b640 1 --2- 172.21.15.16:0/659375986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1930003f30 0x7f1930004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f194259c640 1 --2- 172.21.15.16:0/659375986 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f194415e390 0x7f19300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f1942d9d640 1 --2- 172.21.15.16:0/659375986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1930004d40 0x7f19300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f1941d9b640 1 -- 172.21.15.16:0/659375986 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f194415e390 msgr2=0x7f19300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:39.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f1941d9b640 1 --2- 172.21.15.16:0/659375986 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f194415e390 0x7f19300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f1941d9b640 1 -- 172.21.15.16:0/659375986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1930004d40 msgr2=0x7f19300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:39.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f1941d9b640 1 --2- 172.21.15.16:0/659375986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1930004d40 0x7f19300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.659+0000 7f1941d9b640 1 -- 172.21.15.16:0/659375986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1930003520 con 0x7f1930003f30 2024-09-12T01:34:39.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.660+0000 7f1941d9b640 1 --2- 172.21.15.16:0/659375986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1930003f30 0x7f1930004800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f19440b7a30 tx=0x7f1930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4d9df30446144f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:39.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.660+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c007d70 con 0x7f1930003f30 2024-09-12T01:34:39.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.660+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f192c005ad0 con 0x7f1930003f30 2024-09-12T01:34:39.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.660+0000 7f194159a640 1 -- 172.21.15.16:0/659375986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c007070 con 0x7f1930003f30 2024-09-12T01:34:39.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.660+0000 7f1948cba640 1 -- 172.21.15.16:0/659375986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1930003f30 msgr2=0x7f1930004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.660+0000 7f1948cba640 1 --2- 172.21.15.16:0/659375986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1930003f30 0x7f1930004800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f19440b7a30 tx=0x7f1930005830 comp rx=0 tx=0).stop 2024-09-12T01:34:39.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.660+0000 7f1942d9d640 1 -- 172.21.15.16:0/659375986 reap_dead start 2024-09-12T01:34:39.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.661+0000 7f1948cba640 1 -- 172.21.15.16:0/659375986 shutdown_connections 2024-09-12T01:34:39.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.661+0000 7f1948cba640 1 -- 172.21.15.16:0/659375986 >> 172.21.15.16:0/659375986 conn(0x7f1944073640 msgr2=0x7f1944073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:39.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.661+0000 7f1948cba640 1 -- 172.21.15.16:0/659375986 shutdown_connections 2024-09-12T01:34:39.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.661+0000 7f1948cba640 1 -- 172.21.15.16:0/659375986 wait complete. 2024-09-12T01:34:39.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.662+0000 7f1948cba640 1 Processor -- start 2024-09-12T01:34:39.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.662+0000 7f1948cba640 1 -- start start 2024-09-12T01:34:39.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1948cba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1930003f30 0x7f1944152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1948cba640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f19440ccf50 0x7f19440cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1948cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440cb5a0 0x7f19440cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1948cba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19440ceb20 con 0x7f19440cb5a0 2024-09-12T01:34:39.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1948cba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f194415e770 con 0x7f1930003f30 2024-09-12T01:34:39.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1948cba640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f19440ceca0 con 0x7f19440ccf50 2024-09-12T01:34:39.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1942d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440cb5a0 0x7f19440cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1942d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440cb5a0 0x7f19440cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43374/0 (socket says 172.21.15.16:43374) 2024-09-12T01:34:39.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1942d9d640 1 -- 172.21.15.16:0/2276247775 learned_addr learned my addr 172.21.15.16:0/2276247775 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:39.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1941d9b640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f19440ccf50 0x7f19440cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f194259c640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1930003f30 0x7f1944152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1942d9d640 1 -- 172.21.15.16:0/2276247775 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f19440ccf50 msgr2=0x7f19440cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1942d9d640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f19440ccf50 0x7f19440cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1942d9d640 1 -- 172.21.15.16:0/2276247775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1930003f30 msgr2=0x7f1944152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1942d9d640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1930003f30 0x7f1944152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1942d9d640 1 -- 172.21.15.16:0/2276247775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19440cc240 con 0x7f19440cb5a0 2024-09-12T01:34:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f1942d9d640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440cb5a0 0x7f19440cba20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f19440d1be0 tx=0x7f193c00f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.663+0000 7f193affd640 1 -- 172.21.15.16:0/2276247775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c019070 con 0x7f19440cb5a0 2024-09-12T01:34:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.664+0000 7f193affd640 1 -- 172.21.15.16:0/2276247775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c00f930 con 0x7f19440cb5a0 2024-09-12T01:34:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.664+0000 7f1948cba640 1 -- 172.21.15.16:0/2276247775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19440cc520 con 0x7f19440cb5a0 2024-09-12T01:34:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.664+0000 7f193affd640 1 -- 172.21.15.16:0/2276247775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c003e60 con 0x7f19440cb5a0 2024-09-12T01:34:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.664+0000 7f1948cba640 1 -- 172.21.15.16:0/2276247775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19440ccae0 con 0x7f19440cb5a0 2024-09-12T01:34:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.664+0000 7f1948cba640 1 -- 172.21.15.16:0/2276247775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19440cbf60 con 0x7f19440cb5a0 2024-09-12T01:34:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.665+0000 7f193affd640 1 -- 172.21.15.16:0/2276247775 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f193c01c070 con 0x7f19440cb5a0 2024-09-12T01:34:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.666+0000 7f193affd640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f192406be60 0x7f192406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.666+0000 7f194259c640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f192406be60 0x7f192406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.666+0000 7f193affd640 1 -- 172.21.15.16:0/2276247775 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6273+0+0 (secure 0 0 0) 0x7f193c08b890 con 0x7f19440cb5a0 2024-09-12T01:34:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.666+0000 7f194259c640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f192406be60 0x7f192406e320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1934009ae0 tx=0x7f1934009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.670+0000 7f193affd640 1 -- 172.21.15.16:0/2276247775 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f193c010040 con 0x7f19440cb5a0 2024-09-12T01:34:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:39.884+0000 7f1948cba640 1 -- 172.21.15.16:0/2276247775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f194406d650 con 0x7f19440cb5a0 2024-09-12T01:34:40.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.535+0000 7f193affd640 1 -- 172.21.15.16:0/2276247775 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7f193c059a40 con 0x7f19440cb5a0 2024-09-12T01:34:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.539+0000 7f1948cba640 1 -- 172.21.15.16:0/2276247775 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f192406be60 msgr2=0x7f192406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.539+0000 7f1948cba640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f192406be60 0x7f192406e320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1934009ae0 tx=0x7f1934009190 comp rx=0 tx=0).stop 2024-09-12T01:34:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.539+0000 7f1948cba640 1 -- 172.21.15.16:0/2276247775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440cb5a0 msgr2=0x7f19440cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.539+0000 7f1948cba640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440cb5a0 0x7f19440cba20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f19440d1be0 tx=0x7f193c00f1b0 comp rx=0 tx=0).stop 2024-09-12T01:34:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.539+0000 7f1948cba640 1 -- 172.21.15.16:0/2276247775 shutdown_connections 2024-09-12T01:34:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.539+0000 7f1948cba640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f192406be60 0x7f192406e320 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:40.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.539+0000 7f1948cba640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440cb5a0 0x7f19440cba20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:40.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.539+0000 7f1948cba640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f19440ccf50 0x7f19440cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:40.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.539+0000 7f1948cba640 1 --2- 172.21.15.16:0/2276247775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1930003f30 0x7f1944152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:40.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.539+0000 7f1948cba640 1 -- 172.21.15.16:0/2276247775 >> 172.21.15.16:0/2276247775 conn(0x7f1944073640 msgr2=0x7f19440b9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:40.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.539+0000 7f1948cba640 1 -- 172.21.15.16:0/2276247775 shutdown_connections 2024-09-12T01:34:40.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.539+0000 7f1948cba640 1 -- 172.21.15.16:0/2276247775 wait complete. 2024-09-12T01:34:40.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T01:34:40.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.708+0000 7f54437e4640 1 Processor -- start 2024-09-12T01:34:40.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.709+0000 7f54437e4640 1 -- start start 2024-09-12T01:34:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.709+0000 7f54437e4640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f543c15e600 0x7f543c0ca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.709+0000 7f54437e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f543c0cc8f0 0x7f543c0cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.709+0000 7f54437e4640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f543c0ce5c0 con 0x7f543c0af8c0 2024-09-12T01:34:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.709+0000 7f54437e4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f543c06f0b0 con 0x7f543c0cc8f0 2024-09-12T01:34:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.709+0000 7f54437e4640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f543c068e60 con 0x7f543c15e600 2024-09-12T01:34:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.710+0000 7f5440d7f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f543c0cc8f0 0x7f543c0cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.710+0000 7f5440d7f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f543c0cc8f0 0x7f543c0cfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43384/0 (socket says 172.21.15.16:43384) 2024-09-12T01:34:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.710+0000 7f5440d7f640 1 -- 172.21.15.16:0/2672863333 learned_addr learned my addr 172.21.15.16:0/2672863333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.710+0000 7f5441580640 1 --2- 172.21.15.16:0/2672863333 >> v2:172.21.15.33:3300/0 conn(0x7f543c15e600 0x7f543c0ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.710+0000 7f5440d7f640 1 -- 172.21.15.16:0/2672863333 >> v2:172.21.15.33:3300/0 conn(0x7f543c15e600 msgr2=0x7f543c0ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.710+0000 7f5440d7f640 1 --2- 172.21.15.16:0/2672863333 >> v2:172.21.15.33:3300/0 conn(0x7f543c15e600 0x7f543c0ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.710+0000 7f5440d7f640 1 -- 172.21.15.16:0/2672863333 >> v1:172.21.15.33:6789/0 conn(0x7f543c0af8c0 legacy=0x7f543c0cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.710+0000 7f5440d7f640 1 -- 172.21.15.16:0/2672863333 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543c0d00e0 con 0x7f543c0cc8f0 2024-09-12T01:34:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.711+0000 7f5440d7f640 1 --2- 172.21.15.16:0/2672863333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543c0cc8f0 0x7f543c0cfba0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f543c0707d0 tx=0x7f543002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c5950bd4644774 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.711+0000 7f543bfff640 1 -- 172.21.15.16:0/2672863333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430037070 con 0x7f543c0cc8f0 2024-09-12T01:34:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.711+0000 7f543bfff640 1 -- 172.21.15.16:0/2672863333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5430032440 con 0x7f543c0cc8f0 2024-09-12T01:34:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.711+0000 7f543bfff640 1 -- 172.21.15.16:0/2672863333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430032810 con 0x7f543c0cc8f0 2024-09-12T01:34:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.711+0000 7f54437e4640 1 -- 172.21.15.16:0/2672863333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543c0cc8f0 msgr2=0x7f543c0cfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.711+0000 7f54437e4640 1 --2- 172.21.15.16:0/2672863333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543c0cc8f0 0x7f543c0cfba0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f543c0707d0 tx=0x7f543002a270 comp rx=0 tx=0).stop 2024-09-12T01:34:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.712+0000 7f54437e4640 1 -- 172.21.15.16:0/2672863333 shutdown_connections 2024-09-12T01:34:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.712+0000 7f54437e4640 1 --2- 172.21.15.16:0/2672863333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543c0cc8f0 0x7f543c0cfba0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.712+0000 7f54437e4640 1 --2- 172.21.15.16:0/2672863333 >> v2:172.21.15.33:3300/0 conn(0x7f543c15e600 0x7f543c0ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.712+0000 7f54437e4640 1 -- 172.21.15.16:0/2672863333 >> 172.21.15.16:0/2672863333 conn(0x7f543c0d02e0 msgr2=0x7f543c0d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.712+0000 7f54437e4640 1 -- 172.21.15.16:0/2672863333 shutdown_connections 2024-09-12T01:34:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.712+0000 7f54437e4640 1 -- 172.21.15.16:0/2672863333 wait complete. 2024-09-12T01:34:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.713+0000 7f54437e4640 1 Processor -- start 2024-09-12T01:34:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.713+0000 7f54437e4640 1 -- start start 2024-09-12T01:34:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f54437e4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f543c0af8c0 0x7f543c1551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f54437e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543c0cc8f0 0x7f543c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f54437e4640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f543c15e600 0x7f543c0c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f54437e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f543c0d1180 con 0x7f543c0cc8f0 2024-09-12T01:34:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f54437e4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f543c0d0ea0 con 0x7f543c0af8c0 2024-09-12T01:34:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f54437e4640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f543c073d10 con 0x7f543c15e600 2024-09-12T01:34:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5440d7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543c0cc8f0 0x7f543c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5440d7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543c0cc8f0 0x7f543c0b4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43392/0 (socket says 172.21.15.16:43392) 2024-09-12T01:34:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5440d7f640 1 -- 172.21.15.16:0/818456387 learned_addr learned my addr 172.21.15.16:0/818456387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5441580640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f543c0af8c0 0x7f543c1551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5441d81640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f543c15e600 0x7f543c0c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5440d7f640 1 -- 172.21.15.16:0/818456387 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f543c15e600 msgr2=0x7f543c0c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5440d7f640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f543c15e600 0x7f543c0c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5440d7f640 1 -- 172.21.15.16:0/818456387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f543c0af8c0 msgr2=0x7f543c1551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5440d7f640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f543c0af8c0 0x7f543c1551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5440d7f640 1 -- 172.21.15.16:0/818456387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543c0b3ee0 con 0x7f543c0cc8f0 2024-09-12T01:34:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5441580640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f543c0af8c0 0x7f543c1551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5440d7f640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543c0cc8f0 0x7f543c0b4f80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f543002a2b0 tx=0x7f54300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5439ffb640 1 -- 172.21.15.16:0/818456387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430037040 con 0x7f543c0cc8f0 2024-09-12T01:34:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5439ffb640 1 -- 172.21.15.16:0/818456387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5430042070 con 0x7f543c0cc8f0 2024-09-12T01:34:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f5439ffb640 1 -- 172.21.15.16:0/818456387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430030880 con 0x7f543c0cc8f0 2024-09-12T01:34:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.714+0000 7f54437e4640 1 -- 172.21.15.16:0/818456387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f543c0b41c0 con 0x7f543c0cc8f0 2024-09-12T01:34:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.715+0000 7f54437e4640 1 -- 172.21.15.16:0/818456387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f543c0b4780 con 0x7f543c0cc8f0 2024-09-12T01:34:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.716+0000 7f54437e4640 1 -- 172.21.15.16:0/818456387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f543c15e9e0 con 0x7f543c0cc8f0 2024-09-12T01:34:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.716+0000 7f5439ffb640 1 -- 172.21.15.16:0/818456387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5430030a20 con 0x7f543c0cc8f0 2024-09-12T01:34:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.717+0000 7f5439ffb640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f542806be30 0x7f542806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.717+0000 7f5439ffb640 1 -- 172.21.15.16:0/818456387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6273+0+0 (secure 0 0 0) 0x7f54300ab910 con 0x7f543c0cc8f0 2024-09-12T01:34:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.717+0000 7f5441580640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f542806be30 0x7f542806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.717+0000 7f5441580640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f542806be30 0x7f542806e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f542c0074c0 tx=0x7f542c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.721+0000 7f5439ffb640 1 -- 172.21.15.16:0/818456387 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f543007a090 con 0x7f543c0cc8f0 2024-09-12T01:34:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:40.899+0000 7f54437e4640 1 -- 172.21.15.16:0/818456387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f543c15d5f0 con 0x7f543c0cc8f0 2024-09-12T01:34:41.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.536+0000 7f5439ffb640 1 -- 172.21.15.16:0/818456387 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) ==== 181+0+0 (secure 0 0 0) 0x7f5430079990 con 0x7f543c0cc8f0 2024-09-12T01:34:41.536 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-12T01:34:41.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.540+0000 7f54437e4640 1 -- 172.21.15.16:0/818456387 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f542806be30 msgr2=0x7f542806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:41.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.540+0000 7f54437e4640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f542806be30 0x7f542806e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f542c0074c0 tx=0x7f542c00b040 comp rx=0 tx=0).stop 2024-09-12T01:34:41.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.540+0000 7f54437e4640 1 -- 172.21.15.16:0/818456387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543c0cc8f0 msgr2=0x7f543c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:41.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.540+0000 7f54437e4640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543c0cc8f0 0x7f543c0b4f80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f543002a2b0 tx=0x7f54300046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:41.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.540+0000 7f54437e4640 1 -- 172.21.15.16:0/818456387 shutdown_connections 2024-09-12T01:34:41.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.540+0000 7f54437e4640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f542806be30 0x7f542806e2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:41.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.540+0000 7f54437e4640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f543c15e600 0x7f543c0c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:41.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.540+0000 7f54437e4640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543c0cc8f0 0x7f543c0b4f80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:41.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.540+0000 7f54437e4640 1 --2- 172.21.15.16:0/818456387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f543c0af8c0 0x7f543c1551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:41.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.540+0000 7f54437e4640 1 -- 172.21.15.16:0/818456387 >> 172.21.15.16:0/818456387 conn(0x7f543c0d02e0 msgr2=0x7f543c0734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:41.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.540+0000 7f54437e4640 1 -- 172.21.15.16:0/818456387 shutdown_connections 2024-09-12T01:34:41.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.540+0000 7f54437e4640 1 -- 172.21.15.16:0/818456387 wait complete. 2024-09-12T01:34:41.552 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T01:34:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.662+0000 7f1ac431f640 1 Processor -- start 2024-09-12T01:34:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.662+0000 7f1ac431f640 1 -- start start 2024-09-12T01:34:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.663+0000 7f1ac431f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1abc15cfd0 0x7f1abc070900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.663+0000 7f1ac431f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1abc162eb0 0x7f1abc0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.663+0000 7f1ac431f640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1abc163350 con 0x7f1abc0b4100 2024-09-12T01:34:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.663+0000 7f1ac431f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1abc163ed0 con 0x7f1abc15cfd0 2024-09-12T01:34:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.663+0000 7f1ac431f640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f1abc0b4570 con 0x7f1abc162eb0 2024-09-12T01:34:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.664+0000 7f1ac20bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1abc15cfd0 0x7f1abc070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.664+0000 7f1ac20bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1abc15cfd0 0x7f1abc070900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43394/0 (socket says 172.21.15.16:43394) 2024-09-12T01:34:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.664+0000 7f1ac20bb640 1 -- 172.21.15.16:0/904331770 learned_addr learned my addr 172.21.15.16:0/904331770 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.664+0000 7f1ac20bb640 1 -- 172.21.15.16:0/904331770 >> v2:172.21.15.33:3300/0 conn(0x7f1abc162eb0 msgr2=0x7f1abc0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.664+0000 7f1ac20bb640 1 --2- 172.21.15.16:0/904331770 >> v2:172.21.15.33:3300/0 conn(0x7f1abc162eb0 0x7f1abc0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.664+0000 7f1ac20bb640 1 -- 172.21.15.16:0/904331770 >> v1:172.21.15.33:6789/0 conn(0x7f1abc0b4100 legacy=0x7f1abc1664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:41.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.664+0000 7f1ac20bb640 1 -- 172.21.15.16:0/904331770 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc163630 con 0x7f1abc15cfd0 2024-09-12T01:34:41.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.666+0000 7f1ac20bb640 1 --2- 172.21.15.16:0/904331770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1abc15cfd0 0x7f1abc070900 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1abc06c820 tx=0x7f1ab802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2680cbd289884d24 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.666+0000 7f1ac10b9640 1 -- 172.21.15.16:0/904331770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8037070 con 0x7f1abc15cfd0 2024-09-12T01:34:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.666+0000 7f1ac10b9640 1 -- 172.21.15.16:0/904331770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab8032440 con 0x7f1abc15cfd0 2024-09-12T01:34:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.666+0000 7f1ac10b9640 1 -- 172.21.15.16:0/904331770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8032810 con 0x7f1abc15cfd0 2024-09-12T01:34:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.667+0000 7f1ac431f640 1 -- 172.21.15.16:0/904331770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1abc15cfd0 msgr2=0x7f1abc070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.667+0000 7f1ac431f640 1 --2- 172.21.15.16:0/904331770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1abc15cfd0 0x7f1abc070900 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1abc06c820 tx=0x7f1ab802a1a0 comp rx=0 tx=0).stop 2024-09-12T01:34:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.667+0000 7f1ac431f640 1 -- 172.21.15.16:0/904331770 shutdown_connections 2024-09-12T01:34:41.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.667+0000 7f1ac431f640 1 --2- 172.21.15.16:0/904331770 >> v2:172.21.15.33:3300/0 conn(0x7f1abc162eb0 0x7f1abc0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:41.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.667+0000 7f1ac431f640 1 --2- 172.21.15.16:0/904331770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1abc15cfd0 0x7f1abc070900 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:41.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.667+0000 7f1ac431f640 1 -- 172.21.15.16:0/904331770 >> 172.21.15.16:0/904331770 conn(0x7f1abc067f80 msgr2=0x7f1abc06baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:41.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.668+0000 7f1ac431f640 1 -- 172.21.15.16:0/904331770 shutdown_connections 2024-09-12T01:34:41.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.669+0000 7f1ac431f640 1 -- 172.21.15.16:0/904331770 wait complete. 2024-09-12T01:34:41.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.670+0000 7f1ac431f640 1 Processor -- start 2024-09-12T01:34:41.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.671+0000 7f1ac431f640 1 -- start start 2024-09-12T01:34:41.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.671+0000 7f1ac431f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1abc0b4100 0x7f1abc1f9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:41.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.671+0000 7f1ac431f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1abc15cfd0 0x7f1abc0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:41.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.671+0000 7f1ac431f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc162eb0 0x7f1abc1fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:41.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.671+0000 7f1ac431f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc163bf0 con 0x7f1abc0b4100 2024-09-12T01:34:41.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.671+0000 7f1ac431f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc15d3b0 con 0x7f1abc162eb0 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.671+0000 7f1ac431f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc070e40 con 0x7f1abc15cfd0 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac20bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1abc0b4100 0x7f1abc1f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac20bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1abc0b4100 0x7f1abc1f9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43400/0 (socket says 172.21.15.16:43400) 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac20bb640 1 -- 172.21.15.16:0/3573473077 learned_addr learned my addr 172.21.15.16:0/3573473077 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac18ba640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1abc15cfd0 0x7f1abc0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac28bc640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc162eb0 0x7f1abc1fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac20bb640 1 -- 172.21.15.16:0/3573473077 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1abc15cfd0 msgr2=0x7f1abc0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac20bb640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1abc15cfd0 0x7f1abc0cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac20bb640 1 -- 172.21.15.16:0/3573473077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc162eb0 msgr2=0x7f1abc1fbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac20bb640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc162eb0 0x7f1abc1fbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac20bb640 1 -- 172.21.15.16:0/3573473077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc0cbb60 con 0x7f1abc0b4100 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac28bc640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc162eb0 0x7f1abc1fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac20bb640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1abc0b4100 0x7f1abc1f9280 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8000c00 tx=0x7f1ab80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ab2ffd640 1 -- 172.21.15.16:0/3573473077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8041070 con 0x7f1abc0b4100 2024-09-12T01:34:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ab2ffd640 1 -- 172.21.15.16:0/3573473077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab803c950 con 0x7f1abc0b4100 2024-09-12T01:34:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac431f640 1 -- 172.21.15.16:0/3573473077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc0c67c0 con 0x7f1abc0b4100 2024-09-12T01:34:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ab2ffd640 1 -- 172.21.15.16:0/3573473077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8037040 con 0x7f1abc0b4100 2024-09-12T01:34:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.672+0000 7f1ac431f640 1 -- 172.21.15.16:0/3573473077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc0c7060 con 0x7f1abc0b4100 2024-09-12T01:34:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.673+0000 7f1ac431f640 1 -- 172.21.15.16:0/3573473077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1abc1657e0 con 0x7f1abc0b4100 2024-09-12T01:34:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.674+0000 7f1ab2ffd640 1 -- 172.21.15.16:0/3573473077 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ab8044070 con 0x7f1abc0b4100 2024-09-12T01:34:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.674+0000 7f1ab2ffd640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1aa006be30 0x7f1aa006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.674+0000 7f1ab2ffd640 1 -- 172.21.15.16:0/3573473077 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5835+0+0 (secure 0 0 0) 0x7f1ab80abc30 con 0x7f1abc0b4100 2024-09-12T01:34:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.674+0000 7f1ac18ba640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1aa006be30 0x7f1aa006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.675+0000 7f1ac18ba640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1aa006be30 0x7f1aa006e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0cf8c0 tx=0x7f1aac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.679+0000 7f1ab2ffd640 1 -- 172.21.15.16:0/3573473077 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1ab807a680 con 0x7f1abc0b4100 2024-09-12T01:34:41.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:41.854+0000 7f1ac431f640 1 -- 172.21.15.16:0/3573473077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1abc0b80c0 con 0x7f1abc0b4100 2024-09-12T01:34:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.538+0000 7f1ab2ffd640 1 -- 172.21.15.16:0/3573473077 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) ==== 169+0+0 (secure 0 0 0) 0x7f1ab8079f80 con 0x7f1abc0b4100 2024-09-12T01:34:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-12T01:34:42.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.542+0000 7f1ac431f640 1 -- 172.21.15.16:0/3573473077 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1aa006be30 msgr2=0x7f1aa006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.542+0000 7f1ac431f640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1aa006be30 0x7f1aa006e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0cf8c0 tx=0x7f1aac008040 comp rx=0 tx=0).stop 2024-09-12T01:34:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.542+0000 7f1ac431f640 1 -- 172.21.15.16:0/3573473077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1abc0b4100 msgr2=0x7f1abc1f9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.542+0000 7f1ac431f640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1abc0b4100 0x7f1abc1f9280 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8000c00 tx=0x7f1ab80046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.542+0000 7f1ac431f640 1 -- 172.21.15.16:0/3573473077 shutdown_connections 2024-09-12T01:34:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.542+0000 7f1ac431f640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1aa006be30 0x7f1aa006e2f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.542+0000 7f1ac431f640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc162eb0 0x7f1abc1fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.542+0000 7f1ac431f640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1abc15cfd0 0x7f1abc0cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.542+0000 7f1ac431f640 1 --2- 172.21.15.16:0/3573473077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1abc0b4100 0x7f1abc1f9280 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.542+0000 7f1ac431f640 1 -- 172.21.15.16:0/3573473077 >> 172.21.15.16:0/3573473077 conn(0x7f1abc067f80 msgr2=0x7f1abc0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.542+0000 7f1ac431f640 1 -- 172.21.15.16:0/3573473077 shutdown_connections 2024-09-12T01:34:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.542+0000 7f1ac431f640 1 -- 172.21.15.16:0/3573473077 wait complete. 2024-09-12T01:34:42.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T01:34:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.669+0000 7f4ae4b86640 1 Processor -- start 2024-09-12T01:34:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.669+0000 7f4ae4b86640 1 -- start start 2024-09-12T01:34:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.669+0000 7f4ae4b86640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4ae015e3d0 0x7f4ae01629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.670+0000 7f4ae4b86640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ae00d05a0 0x7f4ae00cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.670+0000 7f4ae4b86640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ae006cca0 con 0x7f4ae00b4530 2024-09-12T01:34:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.670+0000 7f4ae4b86640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4ae00cf6f0 con 0x7f4ae00d05a0 2024-09-12T01:34:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.670+0000 7f4ae4b86640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae0162ee0 con 0x7f4ae015e3d0 2024-09-12T01:34:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.670+0000 7f4aded9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ae00d05a0 0x7f4ae00cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.670+0000 7f4aded9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ae00d05a0 0x7f4ae00cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43412/0 (socket says 172.21.15.16:43412) 2024-09-12T01:34:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.670+0000 7f4aded9d640 1 -- 172.21.15.16:0/4025661726 learned_addr learned my addr 172.21.15.16:0/4025661726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.670+0000 7f4ade59c640 1 --2- 172.21.15.16:0/4025661726 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4ae015e3d0 0x7f4ae01629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.670+0000 7f4aded9d640 1 -- 172.21.15.16:0/4025661726 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4ae015e3d0 msgr2=0x7f4ae01629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.670+0000 7f4aded9d640 1 --2- 172.21.15.16:0/4025661726 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4ae015e3d0 0x7f4ae01629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.670+0000 7f4aded9d640 1 -- 172.21.15.16:0/4025661726 >> v1:172.21.15.33:6789/0 conn(0x7f4ae00b4530 legacy=0x7f4ae00cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.670+0000 7f4aded9d640 1 -- 172.21.15.16:0/4025661726 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae006cf80 con 0x7f4ae00d05a0 2024-09-12T01:34:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.671+0000 7f4aded9d640 1 --2- 172.21.15.16:0/4025661726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae00d05a0 0x7f4ae00cc910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009870 tx=0x7f4ad802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806d8c03ae5292ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.671+0000 7f4add59a640 1 -- 172.21.15.16:0/4025661726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8037070 con 0x7f4ae00d05a0 2024-09-12T01:34:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.671+0000 7f4add59a640 1 -- 172.21.15.16:0/4025661726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad8032440 con 0x7f4ae00d05a0 2024-09-12T01:34:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.671+0000 7f4add59a640 1 -- 172.21.15.16:0/4025661726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8032810 con 0x7f4ae00d05a0 2024-09-12T01:34:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.671+0000 7f4ae4b86640 1 -- 172.21.15.16:0/4025661726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae00d05a0 msgr2=0x7f4ae00cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.671+0000 7f4ae4b86640 1 --2- 172.21.15.16:0/4025661726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae00d05a0 0x7f4ae00cc910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009870 tx=0x7f4ad802a270 comp rx=0 tx=0).stop 2024-09-12T01:34:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.672+0000 7f4ae4b86640 1 -- 172.21.15.16:0/4025661726 shutdown_connections 2024-09-12T01:34:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.672+0000 7f4ae4b86640 1 --2- 172.21.15.16:0/4025661726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae00d05a0 0x7f4ae00cc910 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.672+0000 7f4ae4b86640 1 --2- 172.21.15.16:0/4025661726 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4ae015e3d0 0x7f4ae01629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.672+0000 7f4ae4b86640 1 -- 172.21.15.16:0/4025661726 >> 172.21.15.16:0/4025661726 conn(0x7f4ae006feb0 msgr2=0x7f4ae0070290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.672+0000 7f4ae4b86640 1 -- 172.21.15.16:0/4025661726 shutdown_connections 2024-09-12T01:34:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.672+0000 7f4ae4b86640 1 -- 172.21.15.16:0/4025661726 wait complete. 2024-09-12T01:34:42.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.673+0000 7f4ae4b86640 1 Processor -- start 2024-09-12T01:34:42.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.673+0000 7f4ae4b86640 1 -- start start 2024-09-12T01:34:42.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ae4b86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae00b4530 0x7f4ae0150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ae4b86640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4ae00d05a0 0x7f4ae01e7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ae4b86640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae015e3d0 0x7f4ae01e8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ae4b86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae006d820 con 0x7f4ae00b4530 2024-09-12T01:34:42.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ae4b86640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae006d540 con 0x7f4ae015e3d0 2024-09-12T01:34:42.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ae4b86640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae015e7b0 con 0x7f4ae00d05a0 2024-09-12T01:34:42.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ade59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae00b4530 0x7f4ae0150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ade59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae00b4530 0x7f4ae0150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43422/0 (socket says 172.21.15.16:43422) 2024-09-12T01:34:42.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ade59c640 1 -- 172.21.15.16:0/2600141047 learned_addr learned my addr 172.21.15.16:0/2600141047 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4aded9d640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae015e3d0 0x7f4ae01e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4addd9b640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4ae00d05a0 0x7f4ae01e7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ade59c640 1 -- 172.21.15.16:0/2600141047 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4ae00d05a0 msgr2=0x7f4ae01e7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ade59c640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4ae00d05a0 0x7f4ae01e7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ade59c640 1 -- 172.21.15.16:0/2600141047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae015e3d0 msgr2=0x7f4ae01e8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ade59c640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae015e3d0 0x7f4ae01e8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ade59c640 1 -- 172.21.15.16:0/2600141047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae015f4f0 con 0x7f4ae00b4530 2024-09-12T01:34:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.674+0000 7f4ade59c640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae00b4530 0x7f4ae0150480 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0007e60 tx=0x7f4ad000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.675+0000 7f4ad6ffd640 1 -- 172.21.15.16:0/2600141047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0014620 con 0x7f4ae00b4530 2024-09-12T01:34:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.675+0000 7f4ad6ffd640 1 -- 172.21.15.16:0/2600141047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad0019070 con 0x7f4ae00b4530 2024-09-12T01:34:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.675+0000 7f4ad6ffd640 1 -- 172.21.15.16:0/2600141047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0014a20 con 0x7f4ae00b4530 2024-09-12T01:34:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.675+0000 7f4ae4b86640 1 -- 172.21.15.16:0/2600141047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae015f7d0 con 0x7f4ae00b4530 2024-09-12T01:34:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.675+0000 7f4ae4b86640 1 -- 172.21.15.16:0/2600141047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae00c6ad0 con 0x7f4ae00b4530 2024-09-12T01:34:42.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.676+0000 7f4ae4b86640 1 -- 172.21.15.16:0/2600141047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae006dd40 con 0x7f4ae00b4530 2024-09-12T01:34:42.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.676+0000 7f4ad6ffd640 1 -- 172.21.15.16:0/2600141047 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ad0014bc0 con 0x7f4ae00b4530 2024-09-12T01:34:42.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.677+0000 7f4ad6ffd640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4ac006be30 0x7f4ac006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.677+0000 7f4ad6ffd640 1 -- 172.21.15.16:0/2600141047 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f4ad008b430 con 0x7f4ae00b4530 2024-09-12T01:34:42.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.677+0000 7f4addd9b640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4ac006be30 0x7f4ac006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.677+0000 7f4addd9b640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4ac006be30 0x7f4ac006e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00c8c00 tx=0x7f4ac8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.681+0000 7f4ad6ffd640 1 -- 172.21.15.16:0/2600141047 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4ad0059940 con 0x7f4ae00b4530 2024-09-12T01:34:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.857+0000 7f4ae4b86640 1 -- 172.21.15.16:0/2600141047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f4ae00c6410 con 0x7f4ae00b4530 2024-09-12T01:34:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.861+0000 7f4ad6ffd640 1 -- 172.21.15.16:0/2600141047 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f4ad005dc40 con 0x7f4ae00b4530 2024-09-12T01:34:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.863+0000 7f4ae4b86640 1 -- 172.21.15.16:0/2600141047 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4ac006be30 msgr2=0x7f4ac006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.863+0000 7f4ae4b86640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4ac006be30 0x7f4ac006e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00c8c00 tx=0x7f4ac8009040 comp rx=0 tx=0).stop 2024-09-12T01:34:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.863+0000 7f4ae4b86640 1 -- 172.21.15.16:0/2600141047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae00b4530 msgr2=0x7f4ae0150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.863+0000 7f4ae4b86640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae00b4530 0x7f4ae0150480 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0007e60 tx=0x7f4ad000e4b0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.863+0000 7f4ae4b86640 1 -- 172.21.15.16:0/2600141047 shutdown_connections 2024-09-12T01:34:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.863+0000 7f4ae4b86640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f4ac006be30 0x7f4ac006e2f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.863+0000 7f4ae4b86640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae015e3d0 0x7f4ae01e8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.863+0000 7f4ae4b86640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f4ae00d05a0 0x7f4ae01e7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.863+0000 7f4ae4b86640 1 --2- 172.21.15.16:0/2600141047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae00b4530 0x7f4ae0150480 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.863+0000 7f4ae4b86640 1 -- 172.21.15.16:0/2600141047 >> 172.21.15.16:0/2600141047 conn(0x7f4ae006feb0 msgr2=0x7f4ae015d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.863+0000 7f4ae4b86640 1 -- 172.21.15.16:0/2600141047 shutdown_connections 2024-09-12T01:34:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.863+0000 7f4ae4b86640 1 -- 172.21.15.16:0/2600141047 wait complete. 2024-09-12T01:34:42.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create' 2024-09-12T01:34:42.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:34:42.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:34:42.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.978+0000 7f1b19df8640 1 Processor -- start 2024-09-12T01:34:42.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.978+0000 7f1b19df8640 1 -- start start 2024-09-12T01:34:42.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.979+0000 7f1b19df8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b1409aa30 0x7f1b140d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.979+0000 7f1b19df8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b1409ae10 0x7f1b1406cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.979+0000 7f1b19df8640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1415e7f0 con 0x7f1b14161ef0 2024-09-12T01:34:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.979+0000 7f1b19df8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1b140d1a50 con 0x7f1b1409aa30 2024-09-12T01:34:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.979+0000 7f1b19df8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f1b140d09b0 con 0x7f1b1409ae10 2024-09-12T01:34:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.979+0000 7f1b137fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b1409aa30 0x7f1b140d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.979+0000 7f1b137fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b1409aa30 0x7f1b140d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43428/0 (socket says 172.21.15.16:43428) 2024-09-12T01:34:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.979+0000 7f1b137fe640 1 -- 172.21.15.16:0/1899039903 learned_addr learned my addr 172.21.15.16:0/1899039903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.980+0000 7f1b12ffd640 1 --2- 172.21.15.16:0/1899039903 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b1409ae10 0x7f1b1406cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.980+0000 7f1b137fe640 1 -- 172.21.15.16:0/1899039903 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b1409ae10 msgr2=0x7f1b1406cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:42.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.980+0000 7f1b137fe640 1 --2- 172.21.15.16:0/1899039903 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b1409ae10 0x7f1b1406cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.980+0000 7f1b137fe640 1 -- 172.21.15.16:0/1899039903 >> v1:172.21.15.33:6789/0 conn(0x7f1b14161ef0 legacy=0x7f1b14162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.980+0000 7f1b137fe640 1 -- 172.21.15.16:0/1899039903 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b140d0c90 con 0x7f1b1409aa30 2024-09-12T01:34:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.980+0000 7f1b137fe640 1 --2- 172.21.15.16:0/1899039903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b1409aa30 0x7f1b140d0350 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009870 tx=0x7f1b0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398ecd223640b1d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.981+0000 7f1b127fc640 1 -- 172.21.15.16:0/1899039903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04037070 con 0x7f1b1409aa30 2024-09-12T01:34:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.981+0000 7f1b127fc640 1 -- 172.21.15.16:0/1899039903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b04032440 con 0x7f1b1409aa30 2024-09-12T01:34:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.981+0000 7f1b127fc640 1 -- 172.21.15.16:0/1899039903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04032810 con 0x7f1b1409aa30 2024-09-12T01:34:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.981+0000 7f1b19df8640 1 -- 172.21.15.16:0/1899039903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b1409aa30 msgr2=0x7f1b140d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.981+0000 7f1b19df8640 1 --2- 172.21.15.16:0/1899039903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b1409aa30 0x7f1b140d0350 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009870 tx=0x7f1b0402a270 comp rx=0 tx=0).stop 2024-09-12T01:34:42.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.981+0000 7f1b19df8640 1 -- 172.21.15.16:0/1899039903 shutdown_connections 2024-09-12T01:34:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.981+0000 7f1b19df8640 1 --2- 172.21.15.16:0/1899039903 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b1409ae10 0x7f1b1406cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.981+0000 7f1b19df8640 1 --2- 172.21.15.16:0/1899039903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b1409aa30 0x7f1b140d0350 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.981+0000 7f1b19df8640 1 -- 172.21.15.16:0/1899039903 >> 172.21.15.16:0/1899039903 conn(0x7f1b140b3cf0 msgr2=0x7f1b140b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.981+0000 7f1b19df8640 1 -- 172.21.15.16:0/1899039903 shutdown_connections 2024-09-12T01:34:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.981+0000 7f1b19df8640 1 -- 172.21.15.16:0/1899039903 wait complete. 2024-09-12T01:34:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.982+0000 7f1b19df8640 1 Processor -- start 2024-09-12T01:34:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.983+0000 7f1b19df8640 1 -- start start 2024-09-12T01:34:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.983+0000 7f1b19df8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b1409aa30 0x7f1b1415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.983+0000 7f1b19df8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b1409ae10 0x7f1b14160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.983+0000 7f1b19df8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b14161ef0 0x7f1b14161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.983+0000 7f1b19df8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b140d1530 con 0x7f1b1409ae10 2024-09-12T01:34:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.983+0000 7f1b19df8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b140d1250 con 0x7f1b14161ef0 2024-09-12T01:34:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.983+0000 7f1b19df8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f1b140b2940 con 0x7f1b1409aa30 2024-09-12T01:34:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.983+0000 7f1b12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b1409ae10 0x7f1b14160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.983+0000 7f1b12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b1409ae10 0x7f1b14160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43440/0 (socket says 172.21.15.16:43440) 2024-09-12T01:34:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.983+0000 7f1b12ffd640 1 -- 172.21.15.16:0/1027146567 learned_addr learned my addr 172.21.15.16:0/1027146567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.983+0000 7f1b137fe640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b1409aa30 0x7f1b1415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.983+0000 7f1b13fff640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b14161ef0 0x7f1b14161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.984+0000 7f1b12ffd640 1 -- 172.21.15.16:0/1027146567 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b1409aa30 msgr2=0x7f1b1415b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.984+0000 7f1b12ffd640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b1409aa30 0x7f1b1415b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.984+0000 7f1b12ffd640 1 -- 172.21.15.16:0/1027146567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b14161ef0 msgr2=0x7f1b14161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.984+0000 7f1b12ffd640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b14161ef0 0x7f1b14161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.984+0000 7f1b12ffd640 1 -- 172.21.15.16:0/1027146567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b140c7da0 con 0x7f1b1409ae10 2024-09-12T01:34:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.984+0000 7f1b12ffd640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b1409ae10 0x7f1b14160c10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800d800 tx=0x7f1b0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.984+0000 7f1afbfff640 1 -- 172.21.15.16:0/1027146567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0800b6c0 con 0x7f1b1409ae10 2024-09-12T01:34:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.984+0000 7f1afbfff640 1 -- 172.21.15.16:0/1027146567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b08019070 con 0x7f1b1409ae10 2024-09-12T01:34:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.984+0000 7f1afbfff640 1 -- 172.21.15.16:0/1027146567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0800bac0 con 0x7f1b1409ae10 2024-09-12T01:34:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.984+0000 7f1b19df8640 1 -- 172.21.15.16:0/1027146567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b140c8080 con 0x7f1b1409ae10 2024-09-12T01:34:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.984+0000 7f1b19df8640 1 -- 172.21.15.16:0/1027146567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b140c8640 con 0x7f1b1409ae10 2024-09-12T01:34:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.985+0000 7f1b19df8640 1 -- 172.21.15.16:0/1027146567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b1409b280 con 0x7f1b1409ae10 2024-09-12T01:34:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.986+0000 7f1afbfff640 1 -- 172.21.15.16:0/1027146567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b0800bc60 con 0x7f1b1409ae10 2024-09-12T01:34:42.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.986+0000 7f1afbfff640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1b0006be30 0x7f1b0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.986+0000 7f1afbfff640 1 -- 172.21.15.16:0/1027146567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f1b0808b020 con 0x7f1b1409ae10 2024-09-12T01:34:42.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.986+0000 7f1b137fe640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1b0006be30 0x7f1b0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.987+0000 7f1b137fe640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1b0006be30 0x7f1b0006e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1b04034000 tx=0x7f1b0402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:42.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:42.991+0000 7f1afbfff640 1 -- 172.21.15.16:0/1027146567 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1b08059530 con 0x7f1b1409ae10 2024-09-12T01:34:43.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.239+0000 7f1b19df8640 1 -- 172.21.15.16:0/1027146567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]} v 0) -- 0x7f1b140b8180 con 0x7f1b1409ae10 2024-09-12T01:34:43.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.539+0000 7f1afbfff640 1 -- 172.21.15.16:0/1027146567 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]}]=0 v105) ==== 138+0+0 (secure 0 0 0) 0x7f1b0805d830 con 0x7f1b1409ae10 2024-09-12T01:34:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.542+0000 7f1b19df8640 1 -- 172.21.15.16:0/1027146567 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1b0006be30 msgr2=0x7f1b0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.542+0000 7f1b19df8640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1b0006be30 0x7f1b0006e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1b04034000 tx=0x7f1b0402e000 comp rx=0 tx=0).stop 2024-09-12T01:34:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.542+0000 7f1b19df8640 1 -- 172.21.15.16:0/1027146567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b1409ae10 msgr2=0x7f1b14160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.542+0000 7f1b19df8640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b1409ae10 0x7f1b14160c10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800d800 tx=0x7f1b0800dcc0 comp rx=0 tx=0).stop 2024-09-12T01:34:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.542+0000 7f1b19df8640 1 -- 172.21.15.16:0/1027146567 shutdown_connections 2024-09-12T01:34:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.542+0000 7f1b19df8640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f1b0006be30 0x7f1b0006e2f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.542+0000 7f1b19df8640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b14161ef0 0x7f1b14161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.542+0000 7f1b19df8640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b1409ae10 0x7f1b14160c10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.542+0000 7f1b19df8640 1 --2- 172.21.15.16:0/1027146567 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f1b1409aa30 0x7f1b1415b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.542+0000 7f1b19df8640 1 -- 172.21.15.16:0/1027146567 >> 172.21.15.16:0/1027146567 conn(0x7f1b140b3cf0 msgr2=0x7f1b1406cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.542+0000 7f1b19df8640 1 -- 172.21.15.16:0/1027146567 shutdown_connections 2024-09-12T01:34:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:43.542+0000 7f1b19df8640 1 -- 172.21.15.16:0/1027146567 wait complete. 2024-09-12T01:34:43.556 INFO:tasks.cephfs_test_runner:MDS starts to handle client caps when it enters clientreplay stage. ... ok 2024-09-12T01:34:43.556 INFO:tasks.cephfs_test_runner: 2024-09-12T01:34:43.556 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-12T01:34:43.556 INFO:tasks.cephfs_test_runner:Ran 1 test in 55.943s 2024-09-12T01:34:43.556 INFO:tasks.cephfs_test_runner: 2024-09-12T01:34:43.556 INFO:tasks.cephfs_test_runner:OK 2024-09-12T01:34:43.557 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-12T01:34:43.570 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-12T01:34:43.578 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-12T01:34:43.578 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:34:43.579 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:34:43.614 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=414 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T01:34:43.614 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:34:43.614 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:34:43.670 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=414 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T01:34:43.671 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:34:43.671 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:34:43.690 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1991 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T01:34:43.690 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-12T01:34:43.690 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:34:43.749 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1991 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T01:34:43.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:43.749 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-12T01:34:43.766 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T01:34:43.766 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T01:34:43.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:43.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T01:34:43.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:43.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:34:43.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:43.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-12T01:34:44.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:44.002 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-09-12T01:34:44.021 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T01:34:44.021 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T01:34:44.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:44.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T01:34:44.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:44.093 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T01:34:44.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:34:44.164 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-09-12T01:34:44.260 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-12T01:34:44.270 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T01:34:44.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:34:44.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.376+0000 7f40d3cba640 1 Processor -- start 2024-09-12T01:34:44.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.376+0000 7f40d3cba640 1 -- start start 2024-09-12T01:34:44.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.377+0000 7f40d3cba640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f40cc06dd90 0x7f40cc0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.377+0000 7f40d3cba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f40cc0cb580 0x7f40cc073720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.377+0000 7f40d3cba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40cc06f530 con 0x7f40cc15e8d0 2024-09-12T01:34:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.377+0000 7f40d3cba640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f40cc0cbbb0 con 0x7f40cc0cb580 2024-09-12T01:34:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.377+0000 7f40d3cba640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f40cc0cbe90 con 0x7f40cc06dd90 2024-09-12T01:34:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.377+0000 7f40d2257640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f40cc0cb580 0x7f40cc073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.377+0000 7f40d1a56640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f40cc15e8d0 0x7f40cc0cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58246/0 (socket says 172.21.15.16:58246) 2024-09-12T01:34:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.377+0000 7f40d1a56640 1 -- 172.21.15.16:0/2932213085 learned_addr learned my addr 172.21.15.16:0/2932213085 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.378+0000 7f40d1255640 1 --2- 172.21.15.16:0/2932213085 >> v2:172.21.15.33:3300/0 conn(0x7f40cc06dd90 0x7f40cc0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.378+0000 7f40d2257640 1 -- 172.21.15.16:0/2932213085 >> v2:172.21.15.33:3300/0 conn(0x7f40cc06dd90 msgr2=0x7f40cc0acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.378+0000 7f40d2257640 1 --2- 172.21.15.16:0/2932213085 >> v2:172.21.15.33:3300/0 conn(0x7f40cc06dd90 0x7f40cc0acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.378+0000 7f40d2257640 1 -- 172.21.15.16:0/2932213085 >> v1:172.21.15.16:6789/0 conn(0x7f40cc15e8d0 legacy=0x7f40cc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.378+0000 7f40d2257640 1 -- 172.21.15.16:0/2932213085 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc0ca5f0 con 0x7f40cc0cb580 2024-09-12T01:34:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.378+0000 7f40d1255640 1 --2- 172.21.15.16:0/2932213085 >> v2:172.21.15.33:3300/0 conn(0x7f40cc06dd90 0x7f40cc0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:44.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.378+0000 7f40d2257640 1 --2- 172.21.15.16:0/2932213085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cb580 0x7f40cc073720 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009d90 tx=0x7f40c402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ae9c705e8a982a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:44.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.378+0000 7f40d0a54640 1 -- 172.21.15.16:0/2932213085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4007740 con 0x7f40cc0cb580 2024-09-12T01:34:44.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.378+0000 7f40d0a54640 1 -- 172.21.15.16:0/2932213085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c40078e0 con 0x7f40cc0cb580 2024-09-12T01:34:44.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.378+0000 7f40d0a54640 1 -- 172.21.15.16:0/2932213085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4007cb0 con 0x7f40cc0cb580 2024-09-12T01:34:44.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.379+0000 7f40d3cba640 1 -- 172.21.15.16:0/2932213085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cb580 msgr2=0x7f40cc073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.379+0000 7f40d3cba640 1 --2- 172.21.15.16:0/2932213085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cb580 0x7f40cc073720 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009d90 tx=0x7f40c402a6a0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.379+0000 7f40d3cba640 1 -- 172.21.15.16:0/2932213085 shutdown_connections 2024-09-12T01:34:44.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.379+0000 7f40d3cba640 1 --2- 172.21.15.16:0/2932213085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cb580 0x7f40cc073720 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.379+0000 7f40d3cba640 1 --2- 172.21.15.16:0/2932213085 >> v2:172.21.15.33:3300/0 conn(0x7f40cc06dd90 0x7f40cc0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.379+0000 7f40d3cba640 1 -- 172.21.15.16:0/2932213085 >> 172.21.15.16:0/2932213085 conn(0x7f40cc0cffa0 msgr2=0x7f40cc0d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:44.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.379+0000 7f40d3cba640 1 -- 172.21.15.16:0/2932213085 shutdown_connections 2024-09-12T01:34:44.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.379+0000 7f40d3cba640 1 -- 172.21.15.16:0/2932213085 wait complete. 2024-09-12T01:34:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.380+0000 7f40d3cba640 1 Processor -- start 2024-09-12T01:34:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.380+0000 7f40d3cba640 1 -- start start 2024-09-12T01:34:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.381+0000 7f40d3cba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc06dd90 0x7f40cc0b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.381+0000 7f40d3cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cb580 0x7f40cc1594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.381+0000 7f40d3cba640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f40cc15e8d0 0x7f40cc159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.381+0000 7f40d3cba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc0ce860 con 0x7f40cc0cb580 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.381+0000 7f40d3cba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc0cabb0 con 0x7f40cc06dd90 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.381+0000 7f40d3cba640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc0ce9e0 con 0x7f40cc15e8d0 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.381+0000 7f40d1255640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cb580 0x7f40cc1594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.381+0000 7f40d1255640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cb580 0x7f40cc1594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43464/0 (socket says 172.21.15.16:43464) 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.381+0000 7f40d1255640 1 -- 172.21.15.16:0/3441687666 learned_addr learned my addr 172.21.15.16:0/3441687666 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.381+0000 7f40d2257640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f40cc15e8d0 0x7f40cc159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.381+0000 7f40d1a56640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc06dd90 0x7f40cc0b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.382+0000 7f40d1255640 1 -- 172.21.15.16:0/3441687666 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f40cc15e8d0 msgr2=0x7f40cc159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.382+0000 7f40d1255640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f40cc15e8d0 0x7f40cc159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.382+0000 7f40d1255640 1 -- 172.21.15.16:0/3441687666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc06dd90 msgr2=0x7f40cc0b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.382+0000 7f40d1255640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc06dd90 0x7f40cc0b41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.382+0000 7f40d1255640 1 -- 172.21.15.16:0/3441687666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc0b3d30 con 0x7f40cc0cb580 2024-09-12T01:34:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.382+0000 7f40d1255640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cb580 0x7f40cc1594f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00c7f0 tx=0x7f40bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.382+0000 7f40c27fc640 1 -- 172.21.15.16:0/3441687666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc015070 con 0x7f40cc0cb580 2024-09-12T01:34:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.382+0000 7f40c27fc640 1 -- 172.21.15.16:0/3441687666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40bc0079d0 con 0x7f40cc0cb580 2024-09-12T01:34:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.382+0000 7f40d3cba640 1 -- 172.21.15.16:0/3441687666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc0b2f70 con 0x7f40cc0cb580 2024-09-12T01:34:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.382+0000 7f40c27fc640 1 -- 172.21.15.16:0/3441687666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc007da0 con 0x7f40cc0cb580 2024-09-12T01:34:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.382+0000 7f40d3cba640 1 -- 172.21.15.16:0/3441687666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc0b3530 con 0x7f40cc0cb580 2024-09-12T01:34:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.383+0000 7f40d3cba640 1 -- 172.21.15.16:0/3441687666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40cc0acdd0 con 0x7f40cc0cb580 2024-09-12T01:34:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.383+0000 7f40c27fc640 1 -- 172.21.15.16:0/3441687666 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40bc018070 con 0x7f40cc0cb580 2024-09-12T01:34:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.384+0000 7f40c27fc640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f40b006be30 0x7f40b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.384+0000 7f40c27fc640 1 -- 172.21.15.16:0/3441687666 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f40bc08ac30 con 0x7f40cc0cb580 2024-09-12T01:34:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.384+0000 7f40d1a56640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f40b006be30 0x7f40b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.384+0000 7f40d1a56640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f40b006be30 0x7f40b006e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f40cc15aef0 tx=0x7f40c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.388+0000 7f40c27fc640 1 -- 172.21.15.16:0/3441687666 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f40bc0596f0 con 0x7f40cc0cb580 2024-09-12T01:34:44.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.561+0000 7f40d3cba640 1 -- 172.21.15.16:0/3441687666 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f40cc0af270 con 0x7f40b006be30 2024-09-12T01:34:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.562+0000 7f40c27fc640 1 -- 172.21.15.16:0/3441687666 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13418 (secure 0 0 0) 0x7f40cc0af270 con 0x7f40b006be30 2024-09-12T01:34:44.562 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:34:44.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.564+0000 7f40d3cba640 1 -- 172.21.15.16:0/3441687666 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f40b006be30 msgr2=0x7f40b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.564+0000 7f40d3cba640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f40b006be30 0x7f40b006e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f40cc15aef0 tx=0x7f40c8009040 comp rx=0 tx=0).stop 2024-09-12T01:34:44.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.564+0000 7f40d3cba640 1 -- 172.21.15.16:0/3441687666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cb580 msgr2=0x7f40cc1594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.564+0000 7f40d3cba640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cb580 0x7f40cc1594f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00c7f0 tx=0x7f40bc00cbf0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.564+0000 7f40d3cba640 1 -- 172.21.15.16:0/3441687666 shutdown_connections 2024-09-12T01:34:44.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.564+0000 7f40d3cba640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f40b006be30 0x7f40b006e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.564+0000 7f40d3cba640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f40cc15e8d0 0x7f40cc159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.564+0000 7f40d3cba640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cb580 0x7f40cc1594f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.564+0000 7f40d3cba640 1 --2- 172.21.15.16:0/3441687666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc06dd90 0x7f40cc0b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.564+0000 7f40d3cba640 1 -- 172.21.15.16:0/3441687666 >> 172.21.15.16:0/3441687666 conn(0x7f40cc0cffa0 msgr2=0x7f40cc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:44.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.564+0000 7f40d3cba640 1 -- 172.21.15.16:0/3441687666 shutdown_connections 2024-09-12T01:34:44.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.564+0000 7f40d3cba640 1 -- 172.21.15.16:0/3441687666 wait complete. 2024-09-12T01:34:44.574 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-09-12T01:34:44.490812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":237828,"kb_used_data":4972,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736892,"statfs":{"total":773094113280,"available":772850577408,"internally_reserved":0,"allocated":5091328,"data_stored":2899836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10322,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-73,"num_write_kb":-34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001443"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":208,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T01:34:40.009859+0000","last_change":"2024-09-12T01:33:01.096542+0000","last_active":"2024-09-12T01:34:40.009859+0000","last_peered":"2024-09-12T01:34:40.009859+0000","last_clean":"2024-09-12T01:34:40.009859+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:34:40.009859+0000","last_undegraded":"2024-09-12T01:34:40.009859+0000","last_fullsized":"2024-09-12T01:34:40.009859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_clean_scrub_stamp":"2024-09-12T01:32:58.512553+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:48:58.741813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":810860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":206929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":206459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":213510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":495616,"data_stored":220456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":471040,"data_stored":206437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1089536,"data_stored":809755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:34:44.575 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:34:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.678+0000 7f693bad0640 1 Processor -- start 2024-09-12T01:34:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.678+0000 7f693bad0640 1 -- start start 2024-09-12T01:34:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.679+0000 7f693bad0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69340cfae0 0x7f69340d2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.679+0000 7f693bad0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f693415ebe0 con 0x7f693406e6a0 2024-09-12T01:34:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.679+0000 7f693bad0640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69340d42d0 con 0x7f693415e800 2024-09-12T01:34:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.679+0000 7f693bad0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f69340d3ff0 con 0x7f69340cfae0 2024-09-12T01:34:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.679+0000 7f693a06d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f693406e6a0 0x7f69340d29a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58254/0 (socket says 172.21.15.16:58254) 2024-09-12T01:34:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.679+0000 7f693906b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69340cfae0 0x7f69340d2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.679+0000 7f693a06d640 1 -- 172.21.15.16:0/2297920781 learned_addr learned my addr 172.21.15.16:0/2297920781 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.680+0000 7f693906b640 1 -- 172.21.15.16:0/2297920781 >> v1:172.21.15.33:6789/0 conn(0x7f693415e800 legacy=0x7f69340cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.680+0000 7f693906b640 1 -- 172.21.15.16:0/2297920781 >> v1:172.21.15.16:6789/0 conn(0x7f693406e6a0 legacy=0x7f69340d29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.680+0000 7f693906b640 1 -- 172.21.15.16:0/2297920781 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693406cbb0 con 0x7f69340cfae0 2024-09-12T01:34:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.680+0000 7f693886a640 1 -- 172.21.15.16:0/2297920781 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2549988964 0 0) 0x7f693415ebe0 con 0x7f693406e6a0 2024-09-12T01:34:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.680+0000 7f693906b640 1 --2- 172.21.15.16:0/2297920781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69340cfae0 0x7f69340d2460 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f69240097a0 tx=0x7f692402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0d4b08f5400739 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.681+0000 7f693886a640 1 -- 172.21.15.16:0/2297920781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924037070 con 0x7f69340cfae0 2024-09-12T01:34:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.681+0000 7f693886a640 1 -- 172.21.15.16:0/2297920781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6924032440 con 0x7f69340cfae0 2024-09-12T01:34:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.681+0000 7f693886a640 1 -- 172.21.15.16:0/2297920781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924032810 con 0x7f69340cfae0 2024-09-12T01:34:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.681+0000 7f693bad0640 1 -- 172.21.15.16:0/2297920781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69340cfae0 msgr2=0x7f69340d2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.681+0000 7f693bad0640 1 --2- 172.21.15.16:0/2297920781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69340cfae0 0x7f69340d2460 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f69240097a0 tx=0x7f692402a230 comp rx=0 tx=0).stop 2024-09-12T01:34:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.681+0000 7f693bad0640 1 -- 172.21.15.16:0/2297920781 shutdown_connections 2024-09-12T01:34:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.681+0000 7f693bad0640 1 --2- 172.21.15.16:0/2297920781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69340cfae0 0x7f69340d2460 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.681+0000 7f693bad0640 1 -- 172.21.15.16:0/2297920781 >> 172.21.15.16:0/2297920781 conn(0x7f693409d750 msgr2=0x7f69340d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.681+0000 7f693bad0640 1 -- 172.21.15.16:0/2297920781 shutdown_connections 2024-09-12T01:34:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.681+0000 7f693bad0640 1 -- 172.21.15.16:0/2297920781 wait complete. 2024-09-12T01:34:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.682+0000 7f693bad0640 1 Processor -- start 2024-09-12T01:34:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.682+0000 7f693bad0640 1 -- start start 2024-09-12T01:34:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.683+0000 7f693bad0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f693406e6a0 0x7f693414e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.683+0000 7f693bad0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69340cfae0 0x7f69340b9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.683+0000 7f693bad0640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f693415e800 0x7f693415f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.683+0000 7f693bad0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69340d4e30 con 0x7f693406e6a0 2024-09-12T01:34:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.683+0000 7f693bad0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f693406d170 con 0x7f69340cfae0 2024-09-12T01:34:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.683+0000 7f693bad0640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f69340d30b0 con 0x7f693415e800 2024-09-12T01:34:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.683+0000 7f693986c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f693406e6a0 0x7f693414e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.683+0000 7f693986c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f693406e6a0 0x7f693414e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43486/0 (socket says 172.21.15.16:43486) 2024-09-12T01:34:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.683+0000 7f693986c640 1 -- 172.21.15.16:0/631093376 learned_addr learned my addr 172.21.15.16:0/631093376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f693a06d640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f693415e800 0x7f693415f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f693906b640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69340cfae0 0x7f69340b9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f693986c640 1 -- 172.21.15.16:0/631093376 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f693415e800 msgr2=0x7f693415f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f693986c640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f693415e800 0x7f693415f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f693986c640 1 -- 172.21.15.16:0/631093376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69340cfae0 msgr2=0x7f69340b9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f693986c640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69340cfae0 0x7f69340b9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f693986c640 1 -- 172.21.15.16:0/631093376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69340cd240 con 0x7f693406e6a0 2024-09-12T01:34:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f693a06d640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f693415e800 0x7f693415f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f693906b640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69340cfae0 0x7f69340b9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f693986c640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f693406e6a0 0x7f693414e5a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f693000c400 tx=0x7f693000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f692a7fc640 1 -- 172.21.15.16:0/631093376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930017070 con 0x7f693406e6a0 2024-09-12T01:34:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f692a7fc640 1 -- 172.21.15.16:0/631093376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69300052d0 con 0x7f693406e6a0 2024-09-12T01:34:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f693bad0640 1 -- 172.21.15.16:0/631093376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693414eae0 con 0x7f693406e6a0 2024-09-12T01:34:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f692a7fc640 1 -- 172.21.15.16:0/631093376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930003830 con 0x7f693406e6a0 2024-09-12T01:34:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.684+0000 7f693bad0640 1 -- 172.21.15.16:0/631093376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69340cbad0 con 0x7f693406e6a0 2024-09-12T01:34:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.685+0000 7f693bad0640 1 -- 172.21.15.16:0/631093376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69340b5780 con 0x7f693406e6a0 2024-09-12T01:34:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.685+0000 7f692a7fc640 1 -- 172.21.15.16:0/631093376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f693001a070 con 0x7f693406e6a0 2024-09-12T01:34:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.686+0000 7f692a7fc640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f691806be60 0x7f691806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.686+0000 7f692a7fc640 1 -- 172.21.15.16:0/631093376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f693008a000 con 0x7f693406e6a0 2024-09-12T01:34:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.686+0000 7f693906b640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f691806be60 0x7f691806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.687+0000 7f693906b640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f691806be60 0x7f691806e320 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6924034000 tx=0x7f692402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.690+0000 7f692a7fc640 1 -- 172.21.15.16:0/631093376 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f69340cd240 con 0x7f693406e6a0 2024-09-12T01:34:44.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.861+0000 7f693bad0640 1 -- 172.21.15.16:0/631093376 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f693406baf0 con 0x7f691806be60 2024-09-12T01:34:44.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.862+0000 7f692a7fc640 1 -- 172.21.15.16:0/631093376 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13418 (secure 0 0 0) 0x7f693406baf0 con 0x7f691806be60 2024-09-12T01:34:44.862 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:44.862 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:34:44.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.864+0000 7f693bad0640 1 -- 172.21.15.16:0/631093376 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f691806be60 msgr2=0x7f691806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.864+0000 7f693bad0640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f691806be60 0x7f691806e320 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6924034000 tx=0x7f692402e000 comp rx=0 tx=0).stop 2024-09-12T01:34:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.864+0000 7f693bad0640 1 -- 172.21.15.16:0/631093376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f693406e6a0 msgr2=0x7f693414e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.864+0000 7f693bad0640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f693406e6a0 0x7f693414e5a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f693000c400 tx=0x7f693000c8c0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.864+0000 7f693bad0640 1 -- 172.21.15.16:0/631093376 shutdown_connections 2024-09-12T01:34:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.864+0000 7f693bad0640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f691806be60 0x7f691806e320 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.864+0000 7f693bad0640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f693415e800 0x7f693415f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.864+0000 7f693bad0640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69340cfae0 0x7f69340b9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.864+0000 7f693bad0640 1 --2- 172.21.15.16:0/631093376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f693406e6a0 0x7f693414e5a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.864+0000 7f693bad0640 1 -- 172.21.15.16:0/631093376 >> 172.21.15.16:0/631093376 conn(0x7f693409d750 msgr2=0x7f69340b5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.864+0000 7f693bad0640 1 -- 172.21.15.16:0/631093376 shutdown_connections 2024-09-12T01:34:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.864+0000 7f693bad0640 1 -- 172.21.15.16:0/631093376 wait complete. 2024-09-12T01:34:44.875 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-09-12T01:34:44.490812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":237828,"kb_used_data":4972,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736892,"statfs":{"total":773094113280,"available":772850577408,"internally_reserved":0,"allocated":5091328,"data_stored":2899836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10322,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-73,"num_write_kb":-34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001443"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":208,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T01:34:40.009859+0000","last_change":"2024-09-12T01:33:01.096542+0000","last_active":"2024-09-12T01:34:40.009859+0000","last_peered":"2024-09-12T01:34:40.009859+0000","last_clean":"2024-09-12T01:34:40.009859+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:34:40.009859+0000","last_undegraded":"2024-09-12T01:34:40.009859+0000","last_fullsized":"2024-09-12T01:34:40.009859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_clean_scrub_stamp":"2024-09-12T01:32:58.512553+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:48:58.741813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":810860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":206929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":206459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":213510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":495616,"data_stored":220456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":471040,"data_stored":206437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1089536,"data_stored":809755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:34:44.876 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T01:34:44.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:34:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.978+0000 7f2ae60b7640 1 Processor -- start 2024-09-12T01:34:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.978+0000 7f2ae60b7640 1 -- start start 2024-09-12T01:34:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.979+0000 7f2ae60b7640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2ae00d0350 0x7f2ae006cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.979+0000 7f2ae60b7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae015e7f0 con 0x7f2ae009aa30 2024-09-12T01:34:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.979+0000 7f2ae60b7640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae00d1a50 con 0x7f2ae00b1930 2024-09-12T01:34:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.979+0000 7f2ae60b7640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae00d09b0 con 0x7f2ae00d0350 2024-09-12T01:34:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.979+0000 7f2adf7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2ae009aa30 0x7f2ae009ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58256/0 (socket says 172.21.15.16:58256) 2024-09-12T01:34:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.979+0000 7f2adf7fe640 1 -- 172.21.15.16:0/885981898 learned_addr learned my addr 172.21.15.16:0/885981898 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4030493946 0 0) 0x7f2ae015e7f0 con 0x7f2ae009aa30 2024-09-12T01:34:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2adeffd640 1 --2- 172.21.15.16:0/885981898 >> v2:172.21.15.33:3300/0 conn(0x7f2ae00d0350 0x7f2ae006cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ac8003520 con 0x7f2ae009aa30 2024-09-12T01:34:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3234132835 0 0) 0x7f2ac8003520 con 0x7f2ae009aa30 2024-09-12T01:34:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 >> v2:172.21.15.33:3300/0 conn(0x7f2ae00d0350 msgr2=0x7f2ae006cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 --2- 172.21.15.16:0/885981898 >> v2:172.21.15.33:3300/0 conn(0x7f2ae00d0350 0x7f2ae006cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 >> v1:172.21.15.33:6789/0 conn(0x7f2ae00b1930 legacy=0x7f2ae00b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae00d0c90 con 0x7f2ae009aa30 2024-09-12T01:34:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7f2ad0001670 con 0x7f2ae009aa30 2024-09-12T01:34:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 >> v1:172.21.15.16:6789/0 conn(0x7f2ae009aa30 legacy=0x7f2ae009ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 --2- 172.21.15.16:0/885981898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8003a20 0x7f2ac8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 --2- 172.21.15.16:0/885981898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac8004800 0x7f2ac8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 --2- 172.21.15.16:0/885981898 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2ac80051c0 0x7f2ac800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae00d09b0 con 0x7f2ac8003a20 2024-09-12T01:34:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac800c0f0 con 0x7f2ac8004800 2024-09-12T01:34:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac800c2f0 con 0x7f2ac80051c0 2024-09-12T01:34:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2adf7fe640 1 --2- 172.21.15.16:0/885981898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8003a20 0x7f2ac8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2adeffd640 1 --2- 172.21.15.16:0/885981898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac8004800 0x7f2ac8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2adffff640 1 --2- 172.21.15.16:0/885981898 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2ac80051c0 0x7f2ac800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2adf7fe640 1 -- 172.21.15.16:0/885981898 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2ac80051c0 msgr2=0x7f2ac800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2adf7fe640 1 --2- 172.21.15.16:0/885981898 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2ac80051c0 0x7f2ac800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2adf7fe640 1 -- 172.21.15.16:0/885981898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac8004800 msgr2=0x7f2ac8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2adf7fe640 1 --2- 172.21.15.16:0/885981898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac8004800 0x7f2ac8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2adf7fe640 1 -- 172.21.15.16:0/885981898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac8003520 con 0x7f2ac8003a20 2024-09-12T01:34:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.980+0000 7f2adffff640 1 -- 172.21.15.16:0/885981898 reap_dead start 2024-09-12T01:34:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.981+0000 7f2adf7fe640 1 --2- 172.21.15.16:0/885981898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8003a20 0x7f2ac8003e40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000c210 tx=0x7f2ac8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2967d2a718cf070b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.981+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0007d30 con 0x7f2ac8003a20 2024-09-12T01:34:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.981+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad00058d0 con 0x7f2ac8003a20 2024-09-12T01:34:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.981+0000 7f2ade7fc640 1 -- 172.21.15.16:0/885981898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0005d20 con 0x7f2ac8003a20 2024-09-12T01:34:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.982+0000 7f2ae60b7640 1 -- 172.21.15.16:0/885981898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8003a20 msgr2=0x7f2ac8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.982+0000 7f2ae60b7640 1 --2- 172.21.15.16:0/885981898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8003a20 0x7f2ac8003e40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000c210 tx=0x7f2ac8005830 comp rx=0 tx=0).stop 2024-09-12T01:34:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.982+0000 7f2ae60b7640 1 -- 172.21.15.16:0/885981898 shutdown_connections 2024-09-12T01:34:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.982+0000 7f2ae60b7640 1 --2- 172.21.15.16:0/885981898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8003a20 0x7f2ac8003e40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.982+0000 7f2ae60b7640 1 -- 172.21.15.16:0/885981898 >> 172.21.15.16:0/885981898 conn(0x7f2ae00b3cf0 msgr2=0x7f2ae00b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.982+0000 7f2ae60b7640 1 -- 172.21.15.16:0/885981898 shutdown_connections 2024-09-12T01:34:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.982+0000 7f2ae60b7640 1 -- 172.21.15.16:0/885981898 wait complete. 2024-09-12T01:34:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.983+0000 7f2ae60b7640 1 Processor -- start 2024-09-12T01:34:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.983+0000 7f2ae60b7640 1 -- start start 2024-09-12T01:34:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2ae60b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8003a20 0x7f2ae0160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2ae60b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ae015d440 0x7f2ae015d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2ae60b7640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2ae00c9340 0x7f2ae00c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2ae60b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae00d1530 con 0x7f2ac8003a20 2024-09-12T01:34:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2ae60b7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae00d1250 con 0x7f2ae015d440 2024-09-12T01:34:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2ae60b7640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae009ae10 con 0x7f2ae00c9340 2024-09-12T01:34:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2adf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8003a20 0x7f2ae0160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2adf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8003a20 0x7f2ae0160f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43508/0 (socket says 172.21.15.16:43508) 2024-09-12T01:34:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2adf7fe640 1 -- 172.21.15.16:0/2497904099 learned_addr learned my addr 172.21.15.16:0/2497904099 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2adffff640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2ae00c9340 0x7f2ae00c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2adeffd640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ae015d440 0x7f2ae015d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2adf7fe640 1 -- 172.21.15.16:0/2497904099 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2ae00c9340 msgr2=0x7f2ae00c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2adf7fe640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2ae00c9340 0x7f2ae00c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2adf7fe640 1 -- 172.21.15.16:0/2497904099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ae015d440 msgr2=0x7f2ae015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2adf7fe640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ae015d440 0x7f2ae015d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2adf7fe640 1 -- 172.21.15.16:0/2497904099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae01f7cc0 con 0x7f2ac8003a20 2024-09-12T01:34:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.984+0000 7f2adffff640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2ae00c9340 0x7f2ae00c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.985+0000 7f2adf7fe640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8003a20 0x7f2ae0160f40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2ad002adf0 tx=0x7f2ad0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.985+0000 7f2ac7fff640 1 -- 172.21.15.16:0/2497904099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0000ed0 con 0x7f2ac8003a20 2024-09-12T01:34:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.985+0000 7f2ac7fff640 1 -- 172.21.15.16:0/2497904099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad0041070 con 0x7f2ac8003a20 2024-09-12T01:34:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.985+0000 7f2ae60b7640 1 -- 172.21.15.16:0/2497904099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae01f7fa0 con 0x7f2ac8003a20 2024-09-12T01:34:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.985+0000 7f2ac7fff640 1 -- 172.21.15.16:0/2497904099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0004c10 con 0x7f2ac8003a20 2024-09-12T01:34:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.985+0000 7f2ae60b7640 1 -- 172.21.15.16:0/2497904099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae01f8560 con 0x7f2ac8003a20 2024-09-12T01:34:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.986+0000 7f2ae60b7640 1 -- 172.21.15.16:0/2497904099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae015de00 con 0x7f2ac8003a20 2024-09-12T01:34:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.986+0000 7f2ac7fff640 1 -- 172.21.15.16:0/2497904099 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ad003b070 con 0x7f2ac8003a20 2024-09-12T01:34:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.987+0000 7f2ac7fff640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2acc06c000 0x7f2acc06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.987+0000 7f2ac7fff640 1 -- 172.21.15.16:0/2497904099 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f2ad00aa4c0 con 0x7f2ac8003a20 2024-09-12T01:34:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.987+0000 7f2adeffd640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2acc06c000 0x7f2acc06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.987+0000 7f2adeffd640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2acc06c000 0x7f2acc06e4c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2ae01619b0 tx=0x7f2ad4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:44.991+0000 7f2ac7fff640 1 -- 172.21.15.16:0/2497904099 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2ad0078f90 con 0x7f2ac8003a20 2024-09-12T01:34:45.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.162+0000 7f2ae60b7640 1 -- 172.21.15.16:0/2497904099 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ae009b1a0 con 0x7f2acc06c000 2024-09-12T01:34:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.163+0000 7f2ac7fff640 1 -- 172.21.15.16:0/2497904099 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13418 (secure 0 0 0) 0x7f2ae009b1a0 con 0x7f2acc06c000 2024-09-12T01:34:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:34:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.165+0000 7f2ae60b7640 1 -- 172.21.15.16:0/2497904099 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2acc06c000 msgr2=0x7f2acc06e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.165+0000 7f2ae60b7640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2acc06c000 0x7f2acc06e4c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2ae01619b0 tx=0x7f2ad4008040 comp rx=0 tx=0).stop 2024-09-12T01:34:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.165+0000 7f2ae60b7640 1 -- 172.21.15.16:0/2497904099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8003a20 msgr2=0x7f2ae0160f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.165+0000 7f2ae60b7640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8003a20 0x7f2ae0160f40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2ad002adf0 tx=0x7f2ad0001a40 comp rx=0 tx=0).stop 2024-09-12T01:34:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.165+0000 7f2ae60b7640 1 -- 172.21.15.16:0/2497904099 shutdown_connections 2024-09-12T01:34:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.165+0000 7f2ae60b7640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f2acc06c000 0x7f2acc06e4c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.165+0000 7f2ae60b7640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f2ae00c9340 0x7f2ae00c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.165+0000 7f2ae60b7640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ae015d440 0x7f2ae015d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.165+0000 7f2ae60b7640 1 --2- 172.21.15.16:0/2497904099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8003a20 0x7f2ae0160f40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.165+0000 7f2ae60b7640 1 -- 172.21.15.16:0/2497904099 >> 172.21.15.16:0/2497904099 conn(0x7f2ae00b3cf0 msgr2=0x7f2ae00cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.165+0000 7f2ae60b7640 1 -- 172.21.15.16:0/2497904099 shutdown_connections 2024-09-12T01:34:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.165+0000 7f2ae60b7640 1 -- 172.21.15.16:0/2497904099 wait complete. 2024-09-12T01:34:45.175 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-09-12T01:34:44.490812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":237828,"kb_used_data":4972,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736892,"statfs":{"total":773094113280,"available":772850577408,"internally_reserved":0,"allocated":5091328,"data_stored":2899836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10322,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-73,"num_write_kb":-34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001443"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":208,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T01:34:40.009859+0000","last_change":"2024-09-12T01:33:01.096542+0000","last_active":"2024-09-12T01:34:40.009859+0000","last_peered":"2024-09-12T01:34:40.009859+0000","last_clean":"2024-09-12T01:34:40.009859+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:34:40.009859+0000","last_undegraded":"2024-09-12T01:34:40.009859+0000","last_fullsized":"2024-09-12T01:34:40.009859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_clean_scrub_stamp":"2024-09-12T01:32:58.512553+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:48:58.741813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":810860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":206929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":206459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":213510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":495616,"data_stored":220456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":471040,"data_stored":206437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1089536,"data_stored":809755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:34:45.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:34:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.278+0000 7f375aec9640 1 Processor -- start 2024-09-12T01:34:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.278+0000 7f375aec9640 1 -- start start 2024-09-12T01:34:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.279+0000 7f375aec9640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f37540cfae0 0x7f37540d2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.279+0000 7f375aec9640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f375406e6a0 0x7f37540d29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.279+0000 7f375aec9640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f375415ebe0 con 0x7f375415e800 2024-09-12T01:34:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.279+0000 7f375aec9640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f37540d42d0 con 0x7f375406e6a0 2024-09-12T01:34:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.279+0000 7f375aec9640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f37540d3ff0 con 0x7f37540cfae0 2024-09-12T01:34:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.280+0000 7f3753fff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f37540cfae0 0x7f37540d2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.280+0000 7f3753fff640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f37540cfae0 0x7f37540d2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:47664/0 (socket says 172.21.15.16:47664) 2024-09-12T01:34:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.280+0000 7f3753fff640 1 -- 172.21.15.16:0/2181946207 learned_addr learned my addr 172.21.15.16:0/2181946207 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.280+0000 7f3759466640 1 --2- 172.21.15.16:0/2181946207 >> v2:172.21.15.33:3300/0 conn(0x7f375406e6a0 0x7f37540d29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.280+0000 7f3753fff640 1 -- 172.21.15.16:0/2181946207 >> v2:172.21.15.33:3300/0 conn(0x7f375406e6a0 msgr2=0x7f37540d29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.280+0000 7f3753fff640 1 --2- 172.21.15.16:0/2181946207 >> v2:172.21.15.33:3300/0 conn(0x7f375406e6a0 0x7f37540d29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.280+0000 7f3753fff640 1 -- 172.21.15.16:0/2181946207 >> v1:172.21.15.33:6789/0 conn(0x7f375415e800 legacy=0x7f37540cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.280+0000 7f3753fff640 1 -- 172.21.15.16:0/2181946207 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375406cbb0 con 0x7f37540cfae0 2024-09-12T01:34:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.281+0000 7f3753fff640 1 --2- 172.21.15.16:0/2181946207 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f37540cfae0 0x7f37540d2460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3740009870 tx=0x7f374002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90ea8b13e93cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.281+0000 7f37537fe640 1 -- 172.21.15.16:0/2181946207 <== mon.2 v2:172.21.15.33:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740037070 con 0x7f37540cfae0 2024-09-12T01:34:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.281+0000 7f37537fe640 1 -- 172.21.15.16:0/2181946207 <== mon.2 v2:172.21.15.33:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3740032440 con 0x7f37540cfae0 2024-09-12T01:34:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.281+0000 7f37537fe640 1 -- 172.21.15.16:0/2181946207 <== mon.2 v2:172.21.15.33:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740032810 con 0x7f37540cfae0 2024-09-12T01:34:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.281+0000 7f375aec9640 1 -- 172.21.15.16:0/2181946207 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f37540cfae0 msgr2=0x7f37540d2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.281+0000 7f375aec9640 1 --2- 172.21.15.16:0/2181946207 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f37540cfae0 0x7f37540d2460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3740009870 tx=0x7f374002a270 comp rx=0 tx=0).stop 2024-09-12T01:34:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.281+0000 7f375aec9640 1 -- 172.21.15.16:0/2181946207 shutdown_connections 2024-09-12T01:34:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.282+0000 7f375aec9640 1 --2- 172.21.15.16:0/2181946207 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f37540cfae0 0x7f37540d2460 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.282+0000 7f375aec9640 1 --2- 172.21.15.16:0/2181946207 >> v2:172.21.15.33:3300/0 conn(0x7f375406e6a0 0x7f37540d29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.282+0000 7f375aec9640 1 -- 172.21.15.16:0/2181946207 >> 172.21.15.16:0/2181946207 conn(0x7f375409d750 msgr2=0x7f37540d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.282+0000 7f375aec9640 1 -- 172.21.15.16:0/2181946207 shutdown_connections 2024-09-12T01:34:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.282+0000 7f375aec9640 1 -- 172.21.15.16:0/2181946207 wait complete. 2024-09-12T01:34:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.283+0000 7f375aec9640 1 Processor -- start 2024-09-12T01:34:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.283+0000 7f375aec9640 1 -- start start 2024-09-12T01:34:45.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f375aec9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375406e6a0 0x7f3754161810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:45.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f375aec9640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f37540cfae0 0x7f3754157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:45.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f375aec9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f375415e800 0x7f37540cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:45.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f375aec9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37540d4e30 con 0x7f375406e6a0 2024-09-12T01:34:45.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f375aec9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f375406d170 con 0x7f375415e800 2024-09-12T01:34:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f375aec9640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f37540d2ee0 con 0x7f37540cfae0 2024-09-12T01:34:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f3758c65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375406e6a0 0x7f3754161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f3758c65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375406e6a0 0x7f3754161810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43524/0 (socket says 172.21.15.16:43524) 2024-09-12T01:34:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f3758c65640 1 -- 172.21.15.16:0/334965499 learned_addr learned my addr 172.21.15.16:0/334965499 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f3753fff640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f37540cfae0 0x7f3754157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f3759466640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f375415e800 0x7f37540cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f3758c65640 1 -- 172.21.15.16:0/334965499 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f37540cfae0 msgr2=0x7f3754157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f3758c65640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f37540cfae0 0x7f3754157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f3758c65640 1 -- 172.21.15.16:0/334965499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f375415e800 msgr2=0x7f37540cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f3758c65640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f375415e800 0x7f37540cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f3758c65640 1 -- 172.21.15.16:0/334965499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3754160770 con 0x7f375406e6a0 2024-09-12T01:34:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f3759466640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f375415e800 0x7f37540cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f3758c65640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375406e6a0 0x7f3754161810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f374800c3f0 tx=0x7f374800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.284+0000 7f37517fa640 1 -- 172.21.15.16:0/334965499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37480052d0 con 0x7f375406e6a0 2024-09-12T01:34:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.285+0000 7f37517fa640 1 -- 172.21.15.16:0/334965499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3748005490 con 0x7f375406e6a0 2024-09-12T01:34:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.285+0000 7f37517fa640 1 -- 172.21.15.16:0/334965499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748007260 con 0x7f375406e6a0 2024-09-12T01:34:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.285+0000 7f375aec9640 1 -- 172.21.15.16:0/334965499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3754160a50 con 0x7f375406e6a0 2024-09-12T01:34:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.285+0000 7f375aec9640 1 -- 172.21.15.16:0/334965499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3754161010 con 0x7f375406e6a0 2024-09-12T01:34:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.286+0000 7f375aec9640 1 -- 172.21.15.16:0/334965499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3754068880 con 0x7f375406e6a0 2024-09-12T01:34:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.286+0000 7f37517fa640 1 -- 172.21.15.16:0/334965499 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3748007400 con 0x7f375406e6a0 2024-09-12T01:34:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.287+0000 7f37517fa640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f373806be30 0x7f373806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.287+0000 7f37517fa640 1 -- 172.21.15.16:0/334965499 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f374808a230 con 0x7f375406e6a0 2024-09-12T01:34:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.287+0000 7f3753fff640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f373806be30 0x7f373806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.287+0000 7f3753fff640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f373806be30 0x7f373806e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3740034000 tx=0x7f374002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.291+0000 7f37517fa640 1 -- 172.21.15.16:0/334965499 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3748058d00 con 0x7f375406e6a0 2024-09-12T01:34:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.467+0000 7f375aec9640 1 -- 172.21.15.16:0/334965499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37540cf3d0 con 0x7f375406e6a0 2024-09-12T01:34:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.468+0000 7f37517fa640 1 -- 172.21.15.16:0/334965499 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+13411 (secure 0 0 0) 0x7f3748058600 con 0x7f375406e6a0 2024-09-12T01:34:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:45.468 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"fsid":"cb5903e4-2f7f-485b-9d92-173a6afaa2f8","created":"2024-09-12T01:32:48.216387+0000","modified":"2024-09-12T01:34:42.537702+0000","last_up_change":"2024-09-12T01:32:59.713367+0000","last_in_change":"2024-09-12T01:32:50.336270+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:32:58.512340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a3801af9-d983-47ce-99e7-c99f040f096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6801","nonce":3659932612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6803","nonce":3659932612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6807","nonce":3659932612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3659932612},{"type":"v1","addr":"172.21.15.16:6805","nonce":3659932612}]},"public_addr":"172.21.15.16:6801/3659932612","cluster_addr":"172.21.15.16:6803/3659932612","heartbeat_back_addr":"172.21.15.16:6807/3659932612","heartbeat_front_addr":"172.21.15.16:6805/3659932612","state":["exists","up"]},{"osd":1,"uuid":"8d151a65-9f20-45b0-9c37-b58c1e5748a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6817","nonce":2624815404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6819","nonce":2624815404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6823","nonce":2624815404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2624815404},{"type":"v1","addr":"172.21.15.16:6821","nonce":2624815404}]},"public_addr":"172.21.15.16:6817/2624815404","cluster_addr":"172.21.15.16:6819/2624815404","heartbeat_back_addr":"172.21.15.16:6823/2624815404","heartbeat_front_addr":"172.21.15.16:6821/2624815404","state":["exists","up"]},{"osd":2,"uuid":"6c1c9bd6-738d-45e1-a96c-55d33076e777","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6825","nonce":3738115042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6827","nonce":3738115042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6831","nonce":3738115042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3738115042},{"type":"v1","addr":"172.21.15.16:6829","nonce":3738115042}]},"public_addr":"172.21.15.16:6825/3738115042","cluster_addr":"172.21.15.16:6827/3738115042","heartbeat_back_addr":"172.21.15.16:6831/3738115042","heartbeat_front_addr":"172.21.15.16:6829/3738115042","state":["exists","up"]},{"osd":3,"uuid":"1acf626b-5963-49ef-8a2a-bb551c666d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6809","nonce":1588628599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6811","nonce":1588628599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6815","nonce":1588628599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1588628599},{"type":"v1","addr":"172.21.15.16:6813","nonce":1588628599}]},"public_addr":"172.21.15.16:6809/1588628599","cluster_addr":"172.21.15.16:6811/1588628599","heartbeat_back_addr":"172.21.15.16:6815/1588628599","heartbeat_front_addr":"172.21.15.16:6813/1588628599","state":["exists","up"]},{"osd":4,"uuid":"79396361-0fb0-4aac-a53e-1a532d30243a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6821","nonce":1133469969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6824","nonce":1133469969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6831","nonce":1133469969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1133469969},{"type":"v1","addr":"172.21.15.33:6828","nonce":1133469969}]},"public_addr":"172.21.15.33:6821/1133469969","cluster_addr":"172.21.15.33:6824/1133469969","heartbeat_back_addr":"172.21.15.33:6831/1133469969","heartbeat_front_addr":"172.21.15.33:6828/1133469969","state":["exists","up"]},{"osd":5,"uuid":"b4779b5c-91a9-4a60-a588-39dad1b2b355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6809","nonce":3986213904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6811","nonce":3986213904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6815","nonce":3986213904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3986213904},{"type":"v1","addr":"172.21.15.33:6813","nonce":3986213904}]},"public_addr":"172.21.15.33:6809/3986213904","cluster_addr":"172.21.15.33:6811/3986213904","heartbeat_back_addr":"172.21.15.33:6815/3986213904","heartbeat_front_addr":"172.21.15.33:6813/3986213904","state":["exists","up"]},{"osd":6,"uuid":"49bb9733-38d5-4bd9-bbd0-f4a00d34d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6817","nonce":1471795647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6820","nonce":1471795647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6827","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6829","nonce":1471795647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1471795647},{"type":"v1","addr":"172.21.15.33:6825","nonce":1471795647}]},"public_addr":"172.21.15.33:6817/1471795647","cluster_addr":"172.21.15.33:6820/1471795647","heartbeat_back_addr":"172.21.15.33:6829/1471795647","heartbeat_front_addr":"172.21.15.33:6825/1471795647","state":["exists","up"]},{"osd":7,"uuid":"1d3259b8-06e4-4eb4-8fbc-cf135bd3ce6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6801","nonce":2262537058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6803","nonce":2262537058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6807","nonce":2262537058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2262537058},{"type":"v1","addr":"172.21.15.33:6805","nonce":2262537058}]},"public_addr":"172.21.15.33:6801/2262537058","cluster_addr":"172.21.15.33:6803/2262537058","heartbeat_back_addr":"172.21.15.33:6807/2262537058","heartbeat_front_addr":"172.21.15.33:6805/2262537058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.419678+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.928178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.324832+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.712516+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.703808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.621405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:57.154809+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:32:56.849230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/2095784417":"2024-09-13T01:34:39.533615+0000","172.21.15.16:6834/2095784417":"2024-09-13T01:34:39.533615+0000","172.21.15.16:6837/4214019228":"2024-09-13T01:34:30.165507+0000","172.21.15.16:6836/4214019228":"2024-09-13T01:34:30.165507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:34:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.470+0000 7f375aec9640 1 -- 172.21.15.16:0/334965499 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f373806be30 msgr2=0x7f373806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.470+0000 7f375aec9640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f373806be30 0x7f373806e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3740034000 tx=0x7f374002e000 comp rx=0 tx=0).stop 2024-09-12T01:34:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.470+0000 7f375aec9640 1 -- 172.21.15.16:0/334965499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375406e6a0 msgr2=0x7f3754161810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.470+0000 7f375aec9640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375406e6a0 0x7f3754161810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f374800c3f0 tx=0x7f374800c8b0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.470+0000 7f375aec9640 1 -- 172.21.15.16:0/334965499 shutdown_connections 2024-09-12T01:34:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.470+0000 7f375aec9640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f373806be30 0x7f373806e2f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.470+0000 7f375aec9640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f375415e800 0x7f37540cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.470+0000 7f375aec9640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f37540cfae0 0x7f3754157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.470+0000 7f375aec9640 1 --2- 172.21.15.16:0/334965499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375406e6a0 0x7f3754161810 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:45.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.470+0000 7f375aec9640 1 -- 172.21.15.16:0/334965499 >> 172.21.15.16:0/334965499 conn(0x7f375409d750 msgr2=0x7f37540b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:45.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.470+0000 7f375aec9640 1 -- 172.21.15.16:0/334965499 shutdown_connections 2024-09-12T01:34:45.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:45.470+0000 7f375aec9640 1 -- 172.21.15.16:0/334965499 wait complete. 2024-09-12T01:34:46.482 INFO:tasks.ceph:Scrubbing osd.0 2024-09-12T01:34:46.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-12T01:34:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.603+0000 7f59435f6640 1 Processor -- start 2024-09-12T01:34:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.604+0000 7f59435f6640 1 -- start start 2024-09-12T01:34:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.604+0000 7f59435f6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f593c0b4810 0x7f593c15fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.604+0000 7f59435f6640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f593c1602c0 0x7f593c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.604+0000 7f59435f6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f593c0cced0 con 0x7f593c15c3a0 2024-09-12T01:34:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.604+0000 7f59435f6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f593c160720 con 0x7f593c0b4810 2024-09-12T01:34:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.604+0000 7f59435f6640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f593c0cdae0 con 0x7f593c1602c0 2024-09-12T01:34:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.605+0000 7f5940b91640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f593c0b4810 0x7f593c15fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.605+0000 7f5941392640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f593c15c3a0 0x7f593c09b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48000/0 (socket says 172.21.15.16:48000) 2024-09-12T01:34:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.605+0000 7f5941392640 1 -- 172.21.15.16:0/462956319 learned_addr learned my addr 172.21.15.16:0/462956319 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.605+0000 7f5941b93640 1 --2- 172.21.15.16:0/462956319 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f593c1602c0 0x7f593c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.605+0000 7f5940b91640 1 -- 172.21.15.16:0/462956319 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f593c1602c0 msgr2=0x7f593c0b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.605+0000 7f5940b91640 1 --2- 172.21.15.16:0/462956319 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f593c1602c0 0x7f593c0b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.605+0000 7f5940b91640 1 -- 172.21.15.16:0/462956319 >> v1:172.21.15.16:6789/0 conn(0x7f593c15c3a0 legacy=0x7f593c09b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.605+0000 7f5940b91640 1 -- 172.21.15.16:0/462956319 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c0cd1b0 con 0x7f593c0b4810 2024-09-12T01:34:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.605+0000 7f5933fff640 1 -- 172.21.15.16:0/462956319 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3447719196 0 0) 0x7f593c0cced0 con 0x7f593c15c3a0 2024-09-12T01:34:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.606+0000 7f5940b91640 1 --2- 172.21.15.16:0/462956319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593c0b4810 0x7f593c15fd80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f592c0097a0 tx=0x7f592c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a7d6c7f36d2305 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.606+0000 7f5933fff640 1 -- 172.21.15.16:0/462956319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c037070 con 0x7f593c0b4810 2024-09-12T01:34:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.606+0000 7f5933fff640 1 -- 172.21.15.16:0/462956319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592c032440 con 0x7f593c0b4810 2024-09-12T01:34:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.606+0000 7f5933fff640 1 -- 172.21.15.16:0/462956319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c032810 con 0x7f593c0b4810 2024-09-12T01:34:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.606+0000 7f59435f6640 1 -- 172.21.15.16:0/462956319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593c0b4810 msgr2=0x7f593c15fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.606+0000 7f59435f6640 1 --2- 172.21.15.16:0/462956319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593c0b4810 0x7f593c15fd80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f592c0097a0 tx=0x7f592c02a180 comp rx=0 tx=0).stop 2024-09-12T01:34:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.607+0000 7f59435f6640 1 -- 172.21.15.16:0/462956319 shutdown_connections 2024-09-12T01:34:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.607+0000 7f59435f6640 1 --2- 172.21.15.16:0/462956319 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f593c1602c0 0x7f593c0b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.607+0000 7f59435f6640 1 --2- 172.21.15.16:0/462956319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593c0b4810 0x7f593c15fd80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.607+0000 7f59435f6640 1 -- 172.21.15.16:0/462956319 >> 172.21.15.16:0/462956319 conn(0x7f593c0b3c10 msgr2=0x7f593c0b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.607+0000 7f59435f6640 1 -- 172.21.15.16:0/462956319 shutdown_connections 2024-09-12T01:34:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.607+0000 7f59435f6640 1 -- 172.21.15.16:0/462956319 wait complete. 2024-09-12T01:34:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.608+0000 7f59435f6640 1 Processor -- start 2024-09-12T01:34:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.608+0000 7f59435f6640 1 -- start start 2024-09-12T01:34:46.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f59435f6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f593c0b4810 0x7f593c0c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:46.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f59435f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593c15c3a0 0x7f593c0b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:46.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f59435f6640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f593c1602c0 0x7f593c0c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f59435f6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f593c0cdf70 con 0x7f593c15c3a0 2024-09-12T01:34:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f59435f6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f593c0cd770 con 0x7f593c0b4810 2024-09-12T01:34:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f59435f6640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f593c0b6910 con 0x7f593c1602c0 2024-09-12T01:34:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f5940b91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593c15c3a0 0x7f593c0b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f5940b91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593c15c3a0 0x7f593c0b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41206/0 (socket says 172.21.15.16:41206) 2024-09-12T01:34:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f5940b91640 1 -- 172.21.15.16:0/148199863 learned_addr learned my addr 172.21.15.16:0/148199863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f5941b93640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f593c1602c0 0x7f593c0c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f5941392640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f593c0b4810 0x7f593c0c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f5940b91640 1 -- 172.21.15.16:0/148199863 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f593c1602c0 msgr2=0x7f593c0c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f5940b91640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f593c1602c0 0x7f593c0c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f5940b91640 1 -- 172.21.15.16:0/148199863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f593c0b4810 msgr2=0x7f593c0c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f5940b91640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f593c0b4810 0x7f593c0c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f5940b91640 1 -- 172.21.15.16:0/148199863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c0b6bf0 con 0x7f593c15c3a0 2024-09-12T01:34:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f5940b91640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593c15c3a0 0x7f593c0b5fa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f592c035040 tx=0x7f592c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.609+0000 7f5931ffb640 1 -- 172.21.15.16:0/148199863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c037040 con 0x7f593c15c3a0 2024-09-12T01:34:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.610+0000 7f5931ffb640 1 -- 172.21.15.16:0/148199863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592c030500 con 0x7f593c15c3a0 2024-09-12T01:34:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.610+0000 7f59435f6640 1 -- 172.21.15.16:0/148199863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c0b6ed0 con 0x7f593c15c3a0 2024-09-12T01:34:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.610+0000 7f5931ffb640 1 -- 172.21.15.16:0/148199863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c030910 con 0x7f593c15c3a0 2024-09-12T01:34:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.610+0000 7f59435f6640 1 -- 172.21.15.16:0/148199863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c0b7490 con 0x7f593c15c3a0 2024-09-12T01:34:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.611+0000 7f59435f6640 1 -- 172.21.15.16:0/148199863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f593c069ee0 con 0x7f593c15c3a0 2024-09-12T01:34:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.611+0000 7f5931ffb640 1 -- 172.21.15.16:0/148199863 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f592c044020 con 0x7f593c15c3a0 2024-09-12T01:34:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.612+0000 7f5931ffb640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f592006be30 0x7f592006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.612+0000 7f5931ffb640 1 -- 172.21.15.16:0/148199863 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f592c0aaf50 con 0x7f593c15c3a0 2024-09-12T01:34:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.612+0000 7f5941392640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f592006be30 0x7f592006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.612+0000 7f5931ffb640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] conn(0x7f5920072370 0x7f59200747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.612+0000 7f5931ffb640 1 -- 172.21.15.16:0/148199863 --> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f592c032440 con 0x7f5920072370 2024-09-12T01:34:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.612+0000 7f5931ffb640 1 -- 172.21.15.16:0/148199863 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f592c0ad570 con 0x7f593c15c3a0 2024-09-12T01:34:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.612+0000 7f5941b93640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] conn(0x7f5920072370 0x7f59200747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.612+0000 7f5941392640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f592006be30 0x7f592006e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f59380012e0 tx=0x7f5938001630 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.612+0000 7f5941b93640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] conn(0x7f5920072370 0x7f59200747b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.613+0000 7f5931ffb640 1 -- 172.21.15.16:0/148199863 <== osd.0 v2:172.21.15.16:6800/3659932612 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f592c032440 con 0x7f5920072370 2024-09-12T01:34:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.626+0000 7f59435f6640 1 -- 172.21.15.16:0/148199863 --> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f593c15fd80 con 0x7f5920072370 2024-09-12T01:34:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.626+0000 7f5931ffb640 1 -- 172.21.15.16:0/148199863 <== osd.0 v2:172.21.15.16:6800/3659932612 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f593c15fd80 con 0x7f5920072370 2024-09-12T01:34:46.626 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:34:46.626 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T01:34:46.626 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:34:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.626+0000 7f59435f6640 1 -- 172.21.15.16:0/148199863 >> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] conn(0x7f5920072370 msgr2=0x7f59200747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.626+0000 7f59435f6640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.16:6800/3659932612,v1:172.21.15.16:6801/3659932612] conn(0x7f5920072370 0x7f59200747b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.626+0000 7f59435f6640 1 -- 172.21.15.16:0/148199863 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f592006be30 msgr2=0x7f592006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.627+0000 7f59435f6640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f592006be30 0x7f592006e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f59380012e0 tx=0x7f5938001630 comp rx=0 tx=0).stop 2024-09-12T01:34:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.627+0000 7f59435f6640 1 -- 172.21.15.16:0/148199863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593c15c3a0 msgr2=0x7f593c0b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.627+0000 7f59435f6640 1 --2- 172.21.15.16:0/148199863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593c15c3a0 0x7f593c0b5fa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f592c035040 tx=0x7f592c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.627+0000 7f5941b93640 1 -- 172.21.15.16:0/148199863 reap_dead start 2024-09-12T01:34:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.627+0000 7f59435f6640 1 -- 172.21.15.16:0/148199863 shutdown_connections 2024-09-12T01:34:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.627+0000 7f59435f6640 1 -- 172.21.15.16:0/148199863 >> 172.21.15.16:0/148199863 conn(0x7f593c0b3c10 msgr2=0x7f593c0714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.627+0000 7f59435f6640 1 -- 172.21.15.16:0/148199863 shutdown_connections 2024-09-12T01:34:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.627+0000 7f59435f6640 1 -- 172.21.15.16:0/148199863 wait complete. 2024-09-12T01:34:46.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-12T01:34:46.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.741+0000 7fa81fcdd640 1 Processor -- start 2024-09-12T01:34:46.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.741+0000 7fa81fcdd640 1 -- start start 2024-09-12T01:34:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.742+0000 7fa81fcdd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa81815c080 0x7fa81815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.742+0000 7fa81fcdd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa818070a50 0x7fa8180cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.742+0000 7fa81fcdd640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa81806ceb0 con 0x7fa8180ccd50 2024-09-12T01:34:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.742+0000 7fa81fcdd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa81806d1b0 con 0x7fa81815c080 2024-09-12T01:34:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.742+0000 7fa81fcdd640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa81815f4b0 con 0x7fa818070a50 2024-09-12T01:34:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.742+0000 7fa81da79640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa81815c080 0x7fa81815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.742+0000 7fa81da79640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa81815c080 0x7fa81815eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41212/0 (socket says 172.21.15.16:41212) 2024-09-12T01:34:46.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.742+0000 7fa81da79640 1 -- 172.21.15.16:0/2266677799 learned_addr learned my addr 172.21.15.16:0/2266677799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:46.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.742+0000 7fa81d278640 1 --2- 172.21.15.16:0/2266677799 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa818070a50 0x7fa8180cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:46.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.742+0000 7fa81da79640 1 -- 172.21.15.16:0/2266677799 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa818070a50 msgr2=0x7fa8180cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:46.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.742+0000 7fa81da79640 1 --2- 172.21.15.16:0/2266677799 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa818070a50 0x7fa8180cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.742+0000 7fa81da79640 1 -- 172.21.15.16:0/2266677799 >> v1:172.21.15.33:6789/0 conn(0x7fa8180ccd50 legacy=0x7fa81806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:46.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.742+0000 7fa81da79640 1 -- 172.21.15.16:0/2266677799 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81815f790 con 0x7fa81815c080 2024-09-12T01:34:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.743+0000 7fa81da79640 1 --2- 172.21.15.16:0/2266677799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa81815c080 0x7fa81815eb80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa8140098a0 tx=0x7fa81402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b86f5a184b71042 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.743+0000 7fa81ca77640 1 -- 172.21.15.16:0/2266677799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814037070 con 0x7fa81815c080 2024-09-12T01:34:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.743+0000 7fa81ca77640 1 -- 172.21.15.16:0/2266677799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa814032440 con 0x7fa81815c080 2024-09-12T01:34:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.743+0000 7fa81ca77640 1 -- 172.21.15.16:0/2266677799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814032810 con 0x7fa81815c080 2024-09-12T01:34:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.744+0000 7fa81fcdd640 1 -- 172.21.15.16:0/2266677799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa81815c080 msgr2=0x7fa81815eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.744+0000 7fa81fcdd640 1 --2- 172.21.15.16:0/2266677799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa81815c080 0x7fa81815eb80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa8140098a0 tx=0x7fa81402a290 comp rx=0 tx=0).stop 2024-09-12T01:34:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.744+0000 7fa81fcdd640 1 -- 172.21.15.16:0/2266677799 shutdown_connections 2024-09-12T01:34:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.744+0000 7fa81fcdd640 1 --2- 172.21.15.16:0/2266677799 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa818070a50 0x7fa8180cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.744+0000 7fa81fcdd640 1 --2- 172.21.15.16:0/2266677799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa81815c080 0x7fa81815eb80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.744+0000 7fa81fcdd640 1 -- 172.21.15.16:0/2266677799 >> 172.21.15.16:0/2266677799 conn(0x7fa81806fde0 msgr2=0x7fa8180701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.744+0000 7fa81fcdd640 1 -- 172.21.15.16:0/2266677799 shutdown_connections 2024-09-12T01:34:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.744+0000 7fa81fcdd640 1 -- 172.21.15.16:0/2266677799 wait complete. 2024-09-12T01:34:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.745+0000 7fa81fcdd640 1 Processor -- start 2024-09-12T01:34:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.745+0000 7fa81fcdd640 1 -- start start 2024-09-12T01:34:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81fcdd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa818070a50 0x7fa818154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81fcdd640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa8180ccd50 0x7fa8180cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81fcdd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81815c080 0x7fa818155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81fcdd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa81815c5e0 con 0x7fa818070a50 2024-09-12T01:34:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81fcdd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa81815c460 con 0x7fa81815c080 2024-09-12T01:34:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81fcdd640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fa81815f0c0 con 0x7fa8180ccd50 2024-09-12T01:34:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81da79640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa818070a50 0x7fa818154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81da79640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa818070a50 0x7fa818154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41224/0 (socket says 172.21.15.16:41224) 2024-09-12T01:34:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81da79640 1 -- 172.21.15.16:0/183003455 learned_addr learned my addr 172.21.15.16:0/183003455 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81e27a640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81815c080 0x7fa818155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81d278640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa8180ccd50 0x7fa8180cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81da79640 1 -- 172.21.15.16:0/183003455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa8180ccd50 msgr2=0x7fa8180cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81da79640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa8180ccd50 0x7fa8180cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81da79640 1 -- 172.21.15.16:0/183003455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81815c080 msgr2=0x7fa818155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81da79640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81815c080 0x7fa818155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81da79640 1 -- 172.21.15.16:0/183003455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8180b56c0 con 0x7fa818070a50 2024-09-12T01:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81d278640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa8180ccd50 0x7fa8180cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81da79640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa818070a50 0x7fa818154aa0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa814000c00 tx=0x7fa8140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa80e7fc640 1 -- 172.21.15.16:0/183003455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814041020 con 0x7fa818070a50 2024-09-12T01:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa80e7fc640 1 -- 172.21.15.16:0/183003455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8140306c0 con 0x7fa818070a50 2024-09-12T01:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa81fcdd640 1 -- 172.21.15.16:0/183003455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8180b59a0 con 0x7fa818070a50 2024-09-12T01:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.746+0000 7fa80e7fc640 1 -- 172.21.15.16:0/183003455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814037040 con 0x7fa818070a50 2024-09-12T01:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.747+0000 7fa81fcdd640 1 -- 172.21.15.16:0/183003455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8180c9b60 con 0x7fa818070a50 2024-09-12T01:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.748+0000 7fa81fcdd640 1 -- 172.21.15.16:0/183003455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8180cc810 con 0x7fa818070a50 2024-09-12T01:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.748+0000 7fa80e7fc640 1 -- 172.21.15.16:0/183003455 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa814044020 con 0x7fa818070a50 2024-09-12T01:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.749+0000 7fa80e7fc640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa7fc06be30 0x7fa7fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.749+0000 7fa81d278640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa7fc06be30 0x7fa7fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.749+0000 7fa80e7fc640 1 -- 172.21.15.16:0/183003455 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7fa8140ab450 con 0x7fa818070a50 2024-09-12T01:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.749+0000 7fa81d278640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa7fc06be30 0x7fa7fc06e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa808006fd0 tx=0x7fa808008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.753+0000 7fa80e7fc640 1 -- 172.21.15.16:0/183003455 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa81404b030 con 0x7fa818070a50 2024-09-12T01:34:46.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.929+0000 7fa81fcdd640 1 -- 172.21.15.16:0/183003455 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fa8180ce450 con 0x7fa7fc06be30 2024-09-12T01:34:46.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.929+0000 7fa80e7fc640 1 -- 172.21.15.16:0/183003455 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa8180ce450 con 0x7fa7fc06be30 2024-09-12T01:34:46.929 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 0 to deep-scrub 2024-09-12T01:34:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.931+0000 7fa81fcdd640 1 -- 172.21.15.16:0/183003455 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa7fc06be30 msgr2=0x7fa7fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.931+0000 7fa81fcdd640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa7fc06be30 0x7fa7fc06e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa808006fd0 tx=0x7fa808008040 comp rx=0 tx=0).stop 2024-09-12T01:34:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.931+0000 7fa81fcdd640 1 -- 172.21.15.16:0/183003455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa818070a50 msgr2=0x7fa818154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.931+0000 7fa81fcdd640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa818070a50 0x7fa818154aa0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa814000c00 tx=0x7fa8140046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.931+0000 7fa81fcdd640 1 -- 172.21.15.16:0/183003455 shutdown_connections 2024-09-12T01:34:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.931+0000 7fa81fcdd640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fa7fc06be30 0x7fa7fc06e2f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.931+0000 7fa81fcdd640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81815c080 0x7fa818155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.931+0000 7fa81fcdd640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fa8180ccd50 0x7fa8180cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.931+0000 7fa81fcdd640 1 --2- 172.21.15.16:0/183003455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa818070a50 0x7fa818154aa0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.931+0000 7fa81fcdd640 1 -- 172.21.15.16:0/183003455 >> 172.21.15.16:0/183003455 conn(0x7fa81806fde0 msgr2=0x7fa818160e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.931+0000 7fa81fcdd640 1 -- 172.21.15.16:0/183003455 shutdown_connections 2024-09-12T01:34:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:46.931+0000 7fa81fcdd640 1 -- 172.21.15.16:0/183003455 wait complete. 2024-09-12T01:34:46.942 INFO:tasks.ceph:Scrubbing osd.1 2024-09-12T01:34:46.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-12T01:34:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.049+0000 7f82545af640 1 Processor -- start 2024-09-12T01:34:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.049+0000 7f82545af640 1 -- start start 2024-09-12T01:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.050+0000 7f82545af640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f824c0d1b20 0x7f824c0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.050+0000 7f82545af640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f824c0cfbc0 0x7f824c0b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.050+0000 7f82545af640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f824c0d2d50 con 0x7f824c15e4c0 2024-09-12T01:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.050+0000 7f82545af640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f824c0b5d00 con 0x7f824c0d1b20 2024-09-12T01:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.050+0000 7f82545af640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f824c0d3030 con 0x7f824c0cfbc0 2024-09-12T01:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.050+0000 7f8251b4a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f824c0d1b20 0x7f824c0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.050+0000 7f8251b4a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f824c0d1b20 0x7f824c0cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41232/0 (socket says 172.21.15.16:41232) 2024-09-12T01:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.050+0000 7f8251b4a640 1 -- 172.21.15.16:0/1264473020 learned_addr learned my addr 172.21.15.16:0/1264473020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.050+0000 7f8252b4c640 1 --2- 172.21.15.16:0/1264473020 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f824c0cfbc0 0x7f824c0b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.050+0000 7f8251b4a640 1 -- 172.21.15.16:0/1264473020 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f824c0cfbc0 msgr2=0x7f824c0b7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.050+0000 7f8251b4a640 1 --2- 172.21.15.16:0/1264473020 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f824c0cfbc0 0x7f824c0b7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.050+0000 7f8251b4a640 1 -- 172.21.15.16:0/1264473020 >> v1:172.21.15.33:6789/0 conn(0x7f824c15e4c0 legacy=0x7f824c0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.050+0000 7f8251b4a640 1 -- 172.21.15.16:0/1264473020 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c0d3310 con 0x7f824c0d1b20 2024-09-12T01:34:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.051+0000 7f8251b4a640 1 --2- 172.21.15.16:0/1264473020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824c0d1b20 0x7f824c0cf680 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f823c009870 tx=0x7f823c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3781fbf38f812b1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.051+0000 7f8251349640 1 -- 172.21.15.16:0/1264473020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c037070 con 0x7f824c0d1b20 2024-09-12T01:34:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.051+0000 7f8251349640 1 -- 172.21.15.16:0/1264473020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f823c032440 con 0x7f824c0d1b20 2024-09-12T01:34:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.051+0000 7f8251349640 1 -- 172.21.15.16:0/1264473020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c032810 con 0x7f824c0d1b20 2024-09-12T01:34:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.052+0000 7f82545af640 1 -- 172.21.15.16:0/1264473020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824c0d1b20 msgr2=0x7f824c0cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.052+0000 7f82545af640 1 --2- 172.21.15.16:0/1264473020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824c0d1b20 0x7f824c0cf680 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f823c009870 tx=0x7f823c02a1c0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.052+0000 7f82545af640 1 -- 172.21.15.16:0/1264473020 shutdown_connections 2024-09-12T01:34:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.052+0000 7f82545af640 1 --2- 172.21.15.16:0/1264473020 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f824c0cfbc0 0x7f824c0b7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.052+0000 7f82545af640 1 --2- 172.21.15.16:0/1264473020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824c0d1b20 0x7f824c0cf680 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.052+0000 7f82545af640 1 -- 172.21.15.16:0/1264473020 >> 172.21.15.16:0/1264473020 conn(0x7f824c09d630 msgr2=0x7f824c0d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.052+0000 7f82545af640 1 -- 172.21.15.16:0/1264473020 shutdown_connections 2024-09-12T01:34:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.052+0000 7f82545af640 1 -- 172.21.15.16:0/1264473020 wait complete. 2024-09-12T01:34:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.053+0000 7f82545af640 1 Processor -- start 2024-09-12T01:34:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.053+0000 7f82545af640 1 -- start start 2024-09-12T01:34:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f82545af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824c0cfbc0 0x7f824c15ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f82545af640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f824c0d1b20 0x7f824c15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f82545af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f824c15e4c0 0x7f824c0d0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f82545af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f824c0d29b0 con 0x7f824c0cfbc0 2024-09-12T01:34:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f82545af640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f824c15e8a0 con 0x7f824c15e4c0 2024-09-12T01:34:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f82545af640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f824c0d2b30 con 0x7f824c0d1b20 2024-09-12T01:34:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f825234b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824c0cfbc0 0x7f824c15ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f825234b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824c0cfbc0 0x7f824c15ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41248/0 (socket says 172.21.15.16:41248) 2024-09-12T01:34:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f825234b640 1 -- 172.21.15.16:0/322970303 learned_addr learned my addr 172.21.15.16:0/322970303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f8252b4c640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f824c15e4c0 0x7f824c0d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f8251b4a640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f824c0d1b20 0x7f824c15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f825234b640 1 -- 172.21.15.16:0/322970303 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f824c0d1b20 msgr2=0x7f824c15d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f825234b640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f824c0d1b20 0x7f824c15d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f825234b640 1 -- 172.21.15.16:0/322970303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f824c15e4c0 msgr2=0x7f824c0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f825234b640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f824c15e4c0 0x7f824c0d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f825234b640 1 -- 172.21.15.16:0/322970303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c15dfd0 con 0x7f824c0cfbc0 2024-09-12T01:34:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f825234b640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824c0cfbc0 0x7f824c15ccf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f824800c3f0 tx=0x7f824800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.054+0000 7f8242ffd640 1 -- 172.21.15.16:0/322970303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82480052d0 con 0x7f824c0cfbc0 2024-09-12T01:34:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.055+0000 7f8242ffd640 1 -- 172.21.15.16:0/322970303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8248005490 con 0x7f824c0cfbc0 2024-09-12T01:34:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.055+0000 7f82545af640 1 -- 172.21.15.16:0/322970303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824c1e7420 con 0x7f824c0cfbc0 2024-09-12T01:34:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.055+0000 7f8242ffd640 1 -- 172.21.15.16:0/322970303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248007260 con 0x7f824c0cfbc0 2024-09-12T01:34:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.055+0000 7f82545af640 1 -- 172.21.15.16:0/322970303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824c1e79e0 con 0x7f824c0cfbc0 2024-09-12T01:34:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.055+0000 7f82545af640 1 -- 172.21.15.16:0/322970303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f824c0cef70 con 0x7f824c0cfbc0 2024-09-12T01:34:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.056+0000 7f8242ffd640 1 -- 172.21.15.16:0/322970303 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8248007400 con 0x7f824c0cfbc0 2024-09-12T01:34:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.056+0000 7f8242ffd640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f823006be30 0x7f823006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.057+0000 7f8242ffd640 1 -- 172.21.15.16:0/322970303 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f824808a140 con 0x7f824c0cfbc0 2024-09-12T01:34:47.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.057+0000 7f8251b4a640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f823006be30 0x7f823006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.057+0000 7f8242ffd640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] conn(0x7f8230072370 0x7f82300747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.057+0000 7f8242ffd640 1 -- 172.21.15.16:0/322970303 --> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8248009e50 con 0x7f8230072370 2024-09-12T01:34:47.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.057+0000 7f8242ffd640 1 -- 172.21.15.16:0/322970303 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8248091050 con 0x7f824c0cfbc0 2024-09-12T01:34:47.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.057+0000 7f8252b4c640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] conn(0x7f8230072370 0x7f82300747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.057+0000 7f8251b4a640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f823006be30 0x7f823006e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f823c034000 tx=0x7f823c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.057+0000 7f8252b4c640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] conn(0x7f8230072370 0x7f82300747b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.058+0000 7f8242ffd640 1 -- 172.21.15.16:0/322970303 <== osd.1 v2:172.21.15.16:6816/2624815404 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8248009e50 con 0x7f8230072370 2024-09-12T01:34:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.070+0000 7f82545af640 1 -- 172.21.15.16:0/322970303 --> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f824c15daa0 con 0x7f8230072370 2024-09-12T01:34:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.071+0000 7f8242ffd640 1 -- 172.21.15.16:0/322970303 <== osd.1 v2:172.21.15.16:6816/2624815404 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f824c15daa0 con 0x7f8230072370 2024-09-12T01:34:47.070 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:34:47.070 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T01:34:47.070 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:34:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.071+0000 7f82545af640 1 -- 172.21.15.16:0/322970303 >> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] conn(0x7f8230072370 msgr2=0x7f82300747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.071+0000 7f82545af640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.16:6816/2624815404,v1:172.21.15.16:6817/2624815404] conn(0x7f8230072370 0x7f82300747b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.071+0000 7f82545af640 1 -- 172.21.15.16:0/322970303 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f823006be30 msgr2=0x7f823006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.071+0000 7f82545af640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f823006be30 0x7f823006e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f823c034000 tx=0x7f823c02e000 comp rx=0 tx=0).stop 2024-09-12T01:34:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.071+0000 7f82545af640 1 -- 172.21.15.16:0/322970303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824c0cfbc0 msgr2=0x7f824c15ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.071+0000 7f82545af640 1 --2- 172.21.15.16:0/322970303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824c0cfbc0 0x7f824c15ccf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f824800c3f0 tx=0x7f824800c8b0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.071+0000 7f8252b4c640 1 -- 172.21.15.16:0/322970303 reap_dead start 2024-09-12T01:34:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.071+0000 7f82545af640 1 -- 172.21.15.16:0/322970303 shutdown_connections 2024-09-12T01:34:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.071+0000 7f82545af640 1 -- 172.21.15.16:0/322970303 >> 172.21.15.16:0/322970303 conn(0x7f824c09d630 msgr2=0x7f824c1e6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.071+0000 7f82545af640 1 -- 172.21.15.16:0/322970303 shutdown_connections 2024-09-12T01:34:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.071+0000 7f82545af640 1 -- 172.21.15.16:0/322970303 wait complete. 2024-09-12T01:34:47.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-12T01:34:47.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.184+0000 7fbda5bb7640 1 Processor -- start 2024-09-12T01:34:47.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.184+0000 7fbda5bb7640 1 -- start start 2024-09-12T01:34:47.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbda5bb7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbda00ad8a0 0x7fbda00ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbda5bb7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbda006ea70 con 0x7fbda015e4c0 2024-09-12T01:34:47.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbda5bb7640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbda00b3760 con 0x7fbda00add10 2024-09-12T01:34:47.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbda5bb7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbda00b3a40 con 0x7fbda00ad8a0 2024-09-12T01:34:47.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbd9f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbda015e4c0 0x7fbda006e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48002/0 (socket says 172.21.15.16:48002) 2024-09-12T01:34:47.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbd9f7fe640 1 -- 172.21.15.16:0/206617630 learned_addr learned my addr 172.21.15.16:0/206617630 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:47.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbd9effd640 1 --2- 172.21.15.16:0/206617630 >> v2:172.21.15.16:3300/0 conn(0x7fbda00ad8a0 0x7fbda00ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbd9e7fc640 1 -- 172.21.15.16:0/206617630 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2301783470 0 0) 0x7fbd90001110 con 0x7fbda015e4c0 2024-09-12T01:34:47.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbd9e7fc640 1 -- 172.21.15.16:0/206617630 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd8c003520 con 0x7fbda015e4c0 2024-09-12T01:34:47.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbd9e7fc640 1 -- 172.21.15.16:0/206617630 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1257498808 0 0) 0x7fbda00b3760 con 0x7fbda00add10 2024-09-12T01:34:47.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbd9e7fc640 1 -- 172.21.15.16:0/206617630 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd90001110 con 0x7fbda00add10 2024-09-12T01:34:47.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbd9effd640 1 -- 172.21.15.16:0/206617630 >> v1:172.21.15.33:6789/0 conn(0x7fbda00add10 legacy=0x7fbda00b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbd9effd640 1 -- 172.21.15.16:0/206617630 >> v1:172.21.15.16:6789/0 conn(0x7fbda015e4c0 legacy=0x7fbda006e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.185+0000 7fbd9effd640 1 -- 172.21.15.16:0/206617630 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda00b3d20 con 0x7fbda00ad8a0 2024-09-12T01:34:47.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.186+0000 7fbd9effd640 1 --2- 172.21.15.16:0/206617630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00ad8a0 0x7fbda00ae2b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbd880097a0 tx=0x7fbd8802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b83470c0155292 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.186+0000 7fbd9e7fc640 1 -- 172.21.15.16:0/206617630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88037070 con 0x7fbda00ad8a0 2024-09-12T01:34:47.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.186+0000 7fbd9e7fc640 1 -- 172.21.15.16:0/206617630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd88032440 con 0x7fbda00ad8a0 2024-09-12T01:34:47.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.186+0000 7fbd9e7fc640 1 -- 172.21.15.16:0/206617630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88032810 con 0x7fbda00ad8a0 2024-09-12T01:34:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.187+0000 7fbda5bb7640 1 -- 172.21.15.16:0/206617630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00ad8a0 msgr2=0x7fbda00ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.187+0000 7fbda5bb7640 1 --2- 172.21.15.16:0/206617630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00ad8a0 0x7fbda00ae2b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbd880097a0 tx=0x7fbd8802a1a0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.187+0000 7fbda5bb7640 1 -- 172.21.15.16:0/206617630 shutdown_connections 2024-09-12T01:34:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.187+0000 7fbda5bb7640 1 --2- 172.21.15.16:0/206617630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00ad8a0 0x7fbda00ae2b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.187+0000 7fbda5bb7640 1 -- 172.21.15.16:0/206617630 >> 172.21.15.16:0/206617630 conn(0x7fbda00d03e0 msgr2=0x7fbda00d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.187+0000 7fbda5bb7640 1 -- 172.21.15.16:0/206617630 shutdown_connections 2024-09-12T01:34:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.187+0000 7fbda5bb7640 1 -- 172.21.15.16:0/206617630 wait complete. 2024-09-12T01:34:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.188+0000 7fbda5bb7640 1 Processor -- start 2024-09-12T01:34:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.188+0000 7fbda5bb7640 1 -- start start 2024-09-12T01:34:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbda5bb7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda00ad8a0 0x7fbda0153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbda5bb7640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbda00add10 0x7fbda00cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbda5bb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda015e4c0 0x7fbda0148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbda5bb7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbda00b4f10 con 0x7fbda015e4c0 2024-09-12T01:34:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbda5bb7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbda00b33f0 con 0x7fbda00ad8a0 2024-09-12T01:34:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbda5bb7640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fbda00b5090 con 0x7fbda00add10 2024-09-12T01:34:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbd9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda015e4c0 0x7fbda0148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbd9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda015e4c0 0x7fbda0148f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41264/0 (socket says 172.21.15.16:41264) 2024-09-12T01:34:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbd9ffff640 1 -- 172.21.15.16:0/759548290 learned_addr learned my addr 172.21.15.16:0/759548290 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbd9effd640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbda00add10 0x7fbda00cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbd9f7fe640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda00ad8a0 0x7fbda0153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbd9ffff640 1 -- 172.21.15.16:0/759548290 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbda00add10 msgr2=0x7fbda00cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbd9ffff640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbda00add10 0x7fbda00cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbd9ffff640 1 -- 172.21.15.16:0/759548290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda00ad8a0 msgr2=0x7fbda0153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbd9ffff640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda00ad8a0 0x7fbda0153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.189+0000 7fbd9ffff640 1 -- 172.21.15.16:0/759548290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda00ce1b0 con 0x7fbda015e4c0 2024-09-12T01:34:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.190+0000 7fbd9ffff640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda015e4c0 0x7fbda0148f00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800c7a0 tx=0x7fbd9800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.190+0000 7fbd97fff640 1 -- 172.21.15.16:0/759548290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98003b70 con 0x7fbda015e4c0 2024-09-12T01:34:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.190+0000 7fbd97fff640 1 -- 172.21.15.16:0/759548290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd98003d10 con 0x7fbda015e4c0 2024-09-12T01:34:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.190+0000 7fbd97fff640 1 -- 172.21.15.16:0/759548290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800aa90 con 0x7fbda015e4c0 2024-09-12T01:34:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.190+0000 7fbda5bb7640 1 -- 172.21.15.16:0/759548290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda00ce490 con 0x7fbda015e4c0 2024-09-12T01:34:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.190+0000 7fbda5bb7640 1 -- 172.21.15.16:0/759548290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda00cea50 con 0x7fbda015e4c0 2024-09-12T01:34:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.191+0000 7fbda5bb7640 1 -- 172.21.15.16:0/759548290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda015c150 con 0x7fbda015e4c0 2024-09-12T01:34:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.191+0000 7fbd97fff640 1 -- 172.21.15.16:0/759548290 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd9800ac30 con 0x7fbda015e4c0 2024-09-12T01:34:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.192+0000 7fbd97fff640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbd8006be30 0x7fbd8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.192+0000 7fbd97fff640 1 -- 172.21.15.16:0/759548290 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7fbd9808aba0 con 0x7fbda015e4c0 2024-09-12T01:34:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.192+0000 7fbd9f7fe640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbd8006be30 0x7fbd8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.192+0000 7fbd9f7fe640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbd8006be30 0x7fbd8006e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd900013b0 tx=0x7fbd90007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.196+0000 7fbd97fff640 1 -- 172.21.15.16:0/759548290 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbda015c150 con 0x7fbda015e4c0 2024-09-12T01:34:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.371+0000 7fbda5bb7640 1 -- 172.21.15.16:0/759548290 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fbda00b4800 con 0x7fbd8006be30 2024-09-12T01:34:47.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.372+0000 7fbd97fff640 1 -- 172.21.15.16:0/759548290 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbda00b4800 con 0x7fbd8006be30 2024-09-12T01:34:47.372 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 1 to deep-scrub 2024-09-12T01:34:47.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.374+0000 7fbda5bb7640 1 -- 172.21.15.16:0/759548290 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbd8006be30 msgr2=0x7fbd8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.374+0000 7fbda5bb7640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbd8006be30 0x7fbd8006e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd900013b0 tx=0x7fbd90007d30 comp rx=0 tx=0).stop 2024-09-12T01:34:47.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.374+0000 7fbda5bb7640 1 -- 172.21.15.16:0/759548290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda015e4c0 msgr2=0x7fbda0148f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.374+0000 7fbda5bb7640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda015e4c0 0x7fbda0148f00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800c7a0 tx=0x7fbd9800cc60 comp rx=0 tx=0).stop 2024-09-12T01:34:47.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.374+0000 7fbda5bb7640 1 -- 172.21.15.16:0/759548290 shutdown_connections 2024-09-12T01:34:47.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.374+0000 7fbda5bb7640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fbd8006be30 0x7fbd8006e2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.374+0000 7fbda5bb7640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda015e4c0 0x7fbda0148f00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.374+0000 7fbda5bb7640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fbda00add10 0x7fbda00cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.374+0000 7fbda5bb7640 1 --2- 172.21.15.16:0/759548290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda00ad8a0 0x7fbda0153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.374+0000 7fbda5bb7640 1 -- 172.21.15.16:0/759548290 >> 172.21.15.16:0/759548290 conn(0x7fbda00d03e0 msgr2=0x7fbda00744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:47.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.374+0000 7fbda5bb7640 1 -- 172.21.15.16:0/759548290 shutdown_connections 2024-09-12T01:34:47.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.374+0000 7fbda5bb7640 1 -- 172.21.15.16:0/759548290 wait complete. 2024-09-12T01:34:47.385 INFO:tasks.ceph:Scrubbing osd.2 2024-09-12T01:34:47.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-12T01:34:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.493+0000 7f43b75f8640 1 Processor -- start 2024-09-12T01:34:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.493+0000 7f43b75f8640 1 -- start start 2024-09-12T01:34:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.494+0000 7f43b75f8640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43b015e4d0 0x7f43b00d3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.494+0000 7f43b75f8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43b00cfa90 0x7f43b00cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.494+0000 7f43b75f8640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43b00d1fb0 con 0x7f43b00d0c90 2024-09-12T01:34:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.494+0000 7f43b75f8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f43b00d2290 con 0x7f43b00cfa90 2024-09-12T01:34:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.494+0000 7f43b75f8640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f43b00d2570 con 0x7f43b015e4d0 2024-09-12T01:34:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.494+0000 7f43b4b93640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43b00cfa90 0x7f43b00cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.494+0000 7f43b4b93640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43b00cfa90 0x7f43b00cff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41272/0 (socket says 172.21.15.16:41272) 2024-09-12T01:34:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.494+0000 7f43b4b93640 1 -- 172.21.15.16:0/2576077689 learned_addr learned my addr 172.21.15.16:0/2576077689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.494+0000 7f43b5394640 1 --2- 172.21.15.16:0/2576077689 >> v2:172.21.15.33:3300/0 conn(0x7f43b015e4d0 0x7f43b00d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.494+0000 7f43b4b93640 1 -- 172.21.15.16:0/2576077689 >> v2:172.21.15.33:3300/0 conn(0x7f43b015e4d0 msgr2=0x7f43b00d3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.494+0000 7f43b4b93640 1 --2- 172.21.15.16:0/2576077689 >> v2:172.21.15.33:3300/0 conn(0x7f43b015e4d0 0x7f43b00d3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.494+0000 7f43b4b93640 1 -- 172.21.15.16:0/2576077689 >> v1:172.21.15.33:6789/0 conn(0x7f43b00d0c90 legacy=0x7f43b00d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.494+0000 7f43b4b93640 1 -- 172.21.15.16:0/2576077689 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b00d2850 con 0x7f43b00cfa90 2024-09-12T01:34:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.495+0000 7f43b4b93640 1 --2- 172.21.15.16:0/2576077689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b00cfa90 0x7f43b00cff00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f43b006c670 tx=0x7f43a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf60aafbc9c43c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.495+0000 7f43a7fff640 1 -- 172.21.15.16:0/2576077689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0037070 con 0x7f43b00cfa90 2024-09-12T01:34:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.495+0000 7f43a7fff640 1 -- 172.21.15.16:0/2576077689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a0032440 con 0x7f43b00cfa90 2024-09-12T01:34:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.495+0000 7f43a7fff640 1 -- 172.21.15.16:0/2576077689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0032810 con 0x7f43b00cfa90 2024-09-12T01:34:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.496+0000 7f43b75f8640 1 -- 172.21.15.16:0/2576077689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b00cfa90 msgr2=0x7f43b00cff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.496+0000 7f43b75f8640 1 --2- 172.21.15.16:0/2576077689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b00cfa90 0x7f43b00cff00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f43b006c670 tx=0x7f43a002a1e0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.496+0000 7f43b75f8640 1 -- 172.21.15.16:0/2576077689 shutdown_connections 2024-09-12T01:34:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.496+0000 7f43b75f8640 1 --2- 172.21.15.16:0/2576077689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b00cfa90 0x7f43b00cff00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.496+0000 7f43b75f8640 1 --2- 172.21.15.16:0/2576077689 >> v2:172.21.15.33:3300/0 conn(0x7f43b015e4d0 0x7f43b00d3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.496+0000 7f43b75f8640 1 -- 172.21.15.16:0/2576077689 >> 172.21.15.16:0/2576077689 conn(0x7f43b009d650 msgr2=0x7f43b00d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:47.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.496+0000 7f43b75f8640 1 -- 172.21.15.16:0/2576077689 shutdown_connections 2024-09-12T01:34:47.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.496+0000 7f43b75f8640 1 -- 172.21.15.16:0/2576077689 wait complete. 2024-09-12T01:34:47.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.497+0000 7f43b75f8640 1 Processor -- start 2024-09-12T01:34:47.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.497+0000 7f43b75f8640 1 -- start start 2024-09-12T01:34:47.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b75f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b00cfa90 0x7f43b00a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b75f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b00d0c90 0x7f43b009e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b75f8640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43b015e4d0 0x7f43b01f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b75f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43b015e8b0 con 0x7f43b00d0c90 2024-09-12T01:34:47.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b75f8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43b00d4d80 con 0x7f43b00cfa90 2024-09-12T01:34:47.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b75f8640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f43b015ea30 con 0x7f43b015e4d0 2024-09-12T01:34:47.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b4b93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b00d0c90 0x7f43b009e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b5394640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b00cfa90 0x7f43b00a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b4b93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b00d0c90 0x7f43b009e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41278/0 (socket says 172.21.15.16:41278) 2024-09-12T01:34:47.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b4b93640 1 -- 172.21.15.16:0/3287908717 learned_addr learned my addr 172.21.15.16:0/3287908717 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:47.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b5b95640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43b015e4d0 0x7f43b01f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b4b93640 1 -- 172.21.15.16:0/3287908717 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43b015e4d0 msgr2=0x7f43b01f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b4b93640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f43b015e4d0 0x7f43b01f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b4b93640 1 -- 172.21.15.16:0/3287908717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b00cfa90 msgr2=0x7f43b00a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b4b93640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b00cfa90 0x7f43b00a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b4b93640 1 -- 172.21.15.16:0/3287908717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b009e830 con 0x7f43b00d0c90 2024-09-12T01:34:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b5394640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b00cfa90 0x7f43b00a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.498+0000 7f43b4b93640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b00d0c90 0x7f43b009e0c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f43a002a220 tx=0x7f43a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.499+0000 7f43a5ffb640 1 -- 172.21.15.16:0/3287908717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0037040 con 0x7f43b00d0c90 2024-09-12T01:34:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.499+0000 7f43a5ffb640 1 -- 172.21.15.16:0/3287908717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a0042070 con 0x7f43b00d0c90 2024-09-12T01:34:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.499+0000 7f43a5ffb640 1 -- 172.21.15.16:0/3287908717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0030880 con 0x7f43b00d0c90 2024-09-12T01:34:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.499+0000 7f43b75f8640 1 -- 172.21.15.16:0/3287908717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b009eb10 con 0x7f43b00d0c90 2024-09-12T01:34:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.499+0000 7f43b75f8640 1 -- 172.21.15.16:0/3287908717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b009f0d0 con 0x7f43b00d0c90 2024-09-12T01:34:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.500+0000 7f43b75f8640 1 -- 172.21.15.16:0/3287908717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43b0069110 con 0x7f43b00d0c90 2024-09-12T01:34:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.500+0000 7f43a5ffb640 1 -- 172.21.15.16:0/3287908717 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43a0030a20 con 0x7f43b00d0c90 2024-09-12T01:34:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.501+0000 7f43a5ffb640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f439406be30 0x7f439406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.501+0000 7f43a5ffb640 1 -- 172.21.15.16:0/3287908717 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f43a00ab4c0 con 0x7f43b00d0c90 2024-09-12T01:34:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.501+0000 7f43b5394640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f439406be30 0x7f439406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.501+0000 7f43a5ffb640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] conn(0x7f4394072370 0x7f43940747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.501+0000 7f43a5ffb640 1 -- 172.21.15.16:0/3287908717 --> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f43a000af30 con 0x7f4394072370 2024-09-12T01:34:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.501+0000 7f43a5ffb640 1 -- 172.21.15.16:0/3287908717 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f43a00b2050 con 0x7f43b00d0c90 2024-09-12T01:34:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.501+0000 7f43b5b95640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] conn(0x7f4394072370 0x7f43940747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.501+0000 7f43b5394640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f439406be30 0x7f439406e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009e90 tx=0x7f43ac009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.501+0000 7f43b5b95640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] conn(0x7f4394072370 0x7f43940747b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.502+0000 7f43a5ffb640 1 -- 172.21.15.16:0/3287908717 <== osd.2 v2:172.21.15.16:6824/3738115042 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f43a000af30 con 0x7f4394072370 2024-09-12T01:34:47.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.515+0000 7f43b75f8640 1 -- 172.21.15.16:0/3287908717 --> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f43b00cff00 con 0x7f4394072370 2024-09-12T01:34:47.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.515+0000 7f43a5ffb640 1 -- 172.21.15.16:0/3287908717 <== osd.2 v2:172.21.15.16:6824/3738115042 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f43b00cff00 con 0x7f4394072370 2024-09-12T01:34:47.515 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:34:47.515 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T01:34:47.515 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:34:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.515+0000 7f43b75f8640 1 -- 172.21.15.16:0/3287908717 >> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] conn(0x7f4394072370 msgr2=0x7f43940747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.515+0000 7f43b75f8640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.16:6824/3738115042,v1:172.21.15.16:6825/3738115042] conn(0x7f4394072370 0x7f43940747b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.516+0000 7f43b75f8640 1 -- 172.21.15.16:0/3287908717 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f439406be30 msgr2=0x7f439406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.516+0000 7f43b75f8640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f439406be30 0x7f439406e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009e90 tx=0x7f43ac009190 comp rx=0 tx=0).stop 2024-09-12T01:34:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.516+0000 7f43b75f8640 1 -- 172.21.15.16:0/3287908717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b00d0c90 msgr2=0x7f43b009e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.516+0000 7f43b75f8640 1 --2- 172.21.15.16:0/3287908717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b00d0c90 0x7f43b009e0c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f43a002a220 tx=0x7f43a00046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.516+0000 7f43b5b95640 1 -- 172.21.15.16:0/3287908717 reap_dead start 2024-09-12T01:34:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.516+0000 7f43b75f8640 1 -- 172.21.15.16:0/3287908717 shutdown_connections 2024-09-12T01:34:47.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.516+0000 7f43b75f8640 1 -- 172.21.15.16:0/3287908717 >> 172.21.15.16:0/3287908717 conn(0x7f43b009d650 msgr2=0x7f43b00b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:47.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.516+0000 7f43b75f8640 1 -- 172.21.15.16:0/3287908717 shutdown_connections 2024-09-12T01:34:47.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.516+0000 7f43b75f8640 1 -- 172.21.15.16:0/3287908717 wait complete. 2024-09-12T01:34:47.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-12T01:34:47.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.630+0000 7f51ffc1e640 1 Processor -- start 2024-09-12T01:34:47.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.630+0000 7f51ffc1e640 1 -- start start 2024-09-12T01:34:47.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.631+0000 7f51ffc1e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51f81617b0 0x7f51f80ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.631+0000 7f51ffc1e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f80731c0 con 0x7f51f80adf00 2024-09-12T01:34:47.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.631+0000 7f51ffc1e640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f80b44f0 con 0x7f51f8161b90 2024-09-12T01:34:47.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.631+0000 7f51ffc1e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f51f80b47d0 con 0x7f51f81617b0 2024-09-12T01:34:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.631+0000 7f51fd9ba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51f81617b0 0x7f51f80ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.631+0000 7f51fe1bb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f51f80adf00 0x7f51f80b5590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48012/0 (socket says 172.21.15.16:48012) 2024-09-12T01:34:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.631+0000 7f51fe1bb640 1 -- 172.21.15.16:0/2288050285 learned_addr learned my addr 172.21.15.16:0/2288050285 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:47.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.631+0000 7f51fd9ba640 1 -- 172.21.15.16:0/2288050285 >> v1:172.21.15.33:6789/0 conn(0x7f51f8161b90 legacy=0x7f51f80b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:47.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.631+0000 7f51fd9ba640 1 -- 172.21.15.16:0/2288050285 >> v1:172.21.15.16:6789/0 conn(0x7f51f80adf00 legacy=0x7f51f80b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.631+0000 7f51fd9ba640 1 -- 172.21.15.16:0/2288050285 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f80b4ab0 con 0x7f51f81617b0 2024-09-12T01:34:47.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.632+0000 7f51fd9ba640 1 --2- 172.21.15.16:0/2288050285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f81617b0 0x7f51f80ad9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f51f40097a0 tx=0x7f51f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4382da2579758b30 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.632+0000 7f51fc9b8640 1 -- 172.21.15.16:0/2288050285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4037070 con 0x7f51f81617b0 2024-09-12T01:34:47.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.632+0000 7f51fc9b8640 1 -- 172.21.15.16:0/2288050285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f4032440 con 0x7f51f81617b0 2024-09-12T01:34:47.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.632+0000 7f51fc9b8640 1 -- 172.21.15.16:0/2288050285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4032810 con 0x7f51f81617b0 2024-09-12T01:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.633+0000 7f51ffc1e640 1 -- 172.21.15.16:0/2288050285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f81617b0 msgr2=0x7f51f80ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.633+0000 7f51ffc1e640 1 --2- 172.21.15.16:0/2288050285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f81617b0 0x7f51f80ad9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f51f40097a0 tx=0x7f51f402a270 comp rx=0 tx=0).stop 2024-09-12T01:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.633+0000 7f51ffc1e640 1 -- 172.21.15.16:0/2288050285 shutdown_connections 2024-09-12T01:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.633+0000 7f51ffc1e640 1 --2- 172.21.15.16:0/2288050285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f81617b0 0x7f51f80ad9c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.633+0000 7f51ffc1e640 1 -- 172.21.15.16:0/2288050285 >> 172.21.15.16:0/2288050285 conn(0x7f51f80723c0 msgr2=0x7f51f80727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.633+0000 7f51ffc1e640 1 -- 172.21.15.16:0/2288050285 shutdown_connections 2024-09-12T01:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.633+0000 7f51ffc1e640 1 -- 172.21.15.16:0/2288050285 wait complete. 2024-09-12T01:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.634+0000 7f51ffc1e640 1 Processor -- start 2024-09-12T01:34:47.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.634+0000 7f51ffc1e640 1 -- start start 2024-09-12T01:34:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51ffc1e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f80adf00 0x7f51f814b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51ffc1e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f51f81617b0 0x7f51f8147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51ffc1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f8161b90 0x7f51f8159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51ffc1e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f51f80b4090 con 0x7f51f8161b90 2024-09-12T01:34:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51ffc1e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f51f80b5070 con 0x7f51f80adf00 2024-09-12T01:34:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51ffc1e640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f51f80b4210 con 0x7f51f81617b0 2024-09-12T01:34:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51fe1bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f8161b90 0x7f51f8159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51fe1bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f8161b90 0x7f51f8159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41302/0 (socket says 172.21.15.16:41302) 2024-09-12T01:34:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51fe1bb640 1 -- 172.21.15.16:0/3868208053 learned_addr learned my addr 172.21.15.16:0/3868208053 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51fd9ba640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f80adf00 0x7f51f814b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51fd1b9640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f51f81617b0 0x7f51f8147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51fe1bb640 1 -- 172.21.15.16:0/3868208053 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f51f81617b0 msgr2=0x7f51f8147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51fe1bb640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f51f81617b0 0x7f51f8147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51fe1bb640 1 -- 172.21.15.16:0/3868208053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f80adf00 msgr2=0x7f51f814b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51fe1bb640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f80adf00 0x7f51f814b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51fe1bb640 1 -- 172.21.15.16:0/3868208053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f815c560 con 0x7f51f8161b90 2024-09-12T01:34:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51fd1b9640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f51f81617b0 0x7f51f8147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.635+0000 7f51fe1bb640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f8161b90 0x7f51f8159810 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f51f000cbf0 tx=0x7f51f000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.636+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3868208053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0017070 con 0x7f51f8161b90 2024-09-12T01:34:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.636+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3868208053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f000ace0 con 0x7f51f8161b90 2024-09-12T01:34:47.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.636+0000 7f51ffc1e640 1 -- 172.21.15.16:0/3868208053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f814b640 con 0x7f51f8161b90 2024-09-12T01:34:47.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.636+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3868208053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0012600 con 0x7f51f8161b90 2024-09-12T01:34:47.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.636+0000 7f51ffc1e640 1 -- 172.21.15.16:0/3868208053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f81490e0 con 0x7f51f8161b90 2024-09-12T01:34:47.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.637+0000 7f51ffc1e640 1 -- 172.21.15.16:0/3868208053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f806fd70 con 0x7f51f8161b90 2024-09-12T01:34:47.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.637+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3868208053 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51f001a070 con 0x7f51f8161b90 2024-09-12T01:34:47.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.638+0000 7f51ee7fc640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f51dc06be30 0x7f51dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.638+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3868208053 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f51f008a960 con 0x7f51f8161b90 2024-09-12T01:34:47.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.638+0000 7f51fd9ba640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f51dc06be30 0x7f51dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.638+0000 7f51fd9ba640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f51dc06be30 0x7f51dc06e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f51f4034000 tx=0x7f51f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.642+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3868208053 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f51f806fd70 con 0x7f51f8161b90 2024-09-12T01:34:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.818+0000 7f51ffc1e640 1 -- 172.21.15.16:0/3868208053 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f51f8069fe0 con 0x7f51dc06be30 2024-09-12T01:34:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.818+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3868208053 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f51f8069fe0 con 0x7f51dc06be30 2024-09-12T01:34:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 2 to deep-scrub 2024-09-12T01:34:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.820+0000 7f51ffc1e640 1 -- 172.21.15.16:0/3868208053 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f51dc06be30 msgr2=0x7f51dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.820+0000 7f51ffc1e640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f51dc06be30 0x7f51dc06e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f51f4034000 tx=0x7f51f402e000 comp rx=0 tx=0).stop 2024-09-12T01:34:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.820+0000 7f51ffc1e640 1 -- 172.21.15.16:0/3868208053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f8161b90 msgr2=0x7f51f8159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.820+0000 7f51ffc1e640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f8161b90 0x7f51f8159810 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f51f000cbf0 tx=0x7f51f000a430 comp rx=0 tx=0).stop 2024-09-12T01:34:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.820+0000 7f51ffc1e640 1 -- 172.21.15.16:0/3868208053 shutdown_connections 2024-09-12T01:34:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.820+0000 7f51ffc1e640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f51dc06be30 0x7f51dc06e2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.820+0000 7f51ffc1e640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f8161b90 0x7f51f8159810 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.820+0000 7f51ffc1e640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f51f81617b0 0x7f51f8147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.820+0000 7f51ffc1e640 1 --2- 172.21.15.16:0/3868208053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f80adf00 0x7f51f814b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.820+0000 7f51ffc1e640 1 -- 172.21.15.16:0/3868208053 >> 172.21.15.16:0/3868208053 conn(0x7f51f80723c0 msgr2=0x7f51f80af090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.821+0000 7f51ffc1e640 1 -- 172.21.15.16:0/3868208053 shutdown_connections 2024-09-12T01:34:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.821+0000 7f51ffc1e640 1 -- 172.21.15.16:0/3868208053 wait complete. 2024-09-12T01:34:47.831 INFO:tasks.ceph:Scrubbing osd.3 2024-09-12T01:34:47.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-12T01:34:47.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.938+0000 7f7d709a6640 1 Processor -- start 2024-09-12T01:34:47.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.938+0000 7f7d709a6640 1 -- start start 2024-09-12T01:34:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.939+0000 7f7d709a6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d6c15e770 0x7f7d6c0d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.939+0000 7f7d709a6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7d6c0d3800 0x7f7d6c162130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.939+0000 7f7d709a6640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d6c0b74c0 con 0x7f7d6c162670 2024-09-12T01:34:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.939+0000 7f7d709a6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7d6c0b7df0 con 0x7f7d6c15e770 2024-09-12T01:34:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.939+0000 7f7d709a6640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f7d6c0b77a0 con 0x7f7d6c0d3800 2024-09-12T01:34:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.939+0000 7f7d6a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d6c15e770 0x7f7d6c0d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.940+0000 7f7d6a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d6c15e770 0x7f7d6c0d32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41314/0 (socket says 172.21.15.16:41314) 2024-09-12T01:34:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.940+0000 7f7d6a59c640 1 -- 172.21.15.16:0/2876021989 learned_addr learned my addr 172.21.15.16:0/2876021989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.940+0000 7f7d69d9b640 1 --2- 172.21.15.16:0/2876021989 >> v2:172.21.15.33:3300/0 conn(0x7f7d6c0d3800 0x7f7d6c162130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.940+0000 7f7d6a59c640 1 -- 172.21.15.16:0/2876021989 >> v2:172.21.15.33:3300/0 conn(0x7f7d6c0d3800 msgr2=0x7f7d6c162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.940+0000 7f7d6a59c640 1 --2- 172.21.15.16:0/2876021989 >> v2:172.21.15.33:3300/0 conn(0x7f7d6c0d3800 0x7f7d6c162130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.940+0000 7f7d6a59c640 1 -- 172.21.15.16:0/2876021989 >> v1:172.21.15.33:6789/0 conn(0x7f7d6c162670 legacy=0x7f7d6c0b9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.940+0000 7f7d6a59c640 1 -- 172.21.15.16:0/2876021989 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c0b7a80 con 0x7f7d6c15e770 2024-09-12T01:34:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.940+0000 7f7d6a59c640 1 --2- 172.21.15.16:0/2876021989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15e770 0x7f7d6c0d32c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0b6eb0 tx=0x7f7d6402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad16d77138dbd866 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.941+0000 7f7d6959a640 1 -- 172.21.15.16:0/2876021989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64036070 con 0x7f7d6c15e770 2024-09-12T01:34:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.941+0000 7f7d6959a640 1 -- 172.21.15.16:0/2876021989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d64031600 con 0x7f7d6c15e770 2024-09-12T01:34:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.941+0000 7f7d6959a640 1 -- 172.21.15.16:0/2876021989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d640319d0 con 0x7f7d6c15e770 2024-09-12T01:34:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.941+0000 7f7d709a6640 1 -- 172.21.15.16:0/2876021989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15e770 msgr2=0x7f7d6c0d32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.941+0000 7f7d709a6640 1 --2- 172.21.15.16:0/2876021989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15e770 0x7f7d6c0d32c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0b6eb0 tx=0x7f7d6402a6f0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.941+0000 7f7d709a6640 1 -- 172.21.15.16:0/2876021989 shutdown_connections 2024-09-12T01:34:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.941+0000 7f7d709a6640 1 --2- 172.21.15.16:0/2876021989 >> v2:172.21.15.33:3300/0 conn(0x7f7d6c0d3800 0x7f7d6c162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.941+0000 7f7d709a6640 1 --2- 172.21.15.16:0/2876021989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15e770 0x7f7d6c0d32c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.941+0000 7f7d709a6640 1 -- 172.21.15.16:0/2876021989 >> 172.21.15.16:0/2876021989 conn(0x7f7d6c09d660 msgr2=0x7f7d6c0d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.941+0000 7f7d709a6640 1 -- 172.21.15.16:0/2876021989 shutdown_connections 2024-09-12T01:34:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.941+0000 7f7d709a6640 1 -- 172.21.15.16:0/2876021989 wait complete. 2024-09-12T01:34:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.943+0000 7f7d709a6640 1 Processor -- start 2024-09-12T01:34:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.943+0000 7f7d709a6640 1 -- start start 2024-09-12T01:34:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.943+0000 7f7d709a6640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f7d6c0d3800 0x7f7d6c1573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.943+0000 7f7d709a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15e770 0x7f7d6c153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.943+0000 7f7d709a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c162670 0x7f7d6c0ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d709a6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c0ba190 con 0x7f7d6c15e770 2024-09-12T01:34:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d709a6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c0d53d0 con 0x7f7d6c162670 2024-09-12T01:34:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d709a6640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6c0ba310 con 0x7f7d6c0d3800 2024-09-12T01:34:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d69d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15e770 0x7f7d6c153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d69d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15e770 0x7f7d6c153af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41326/0 (socket says 172.21.15.16:41326) 2024-09-12T01:34:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d69d9b640 1 -- 172.21.15.16:0/1226388059 learned_addr learned my addr 172.21.15.16:0/1226388059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d6a59c640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f7d6c0d3800 0x7f7d6c1573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d6ad9d640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c162670 0x7f7d6c0ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d69d9b640 1 -- 172.21.15.16:0/1226388059 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f7d6c0d3800 msgr2=0x7f7d6c1573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d69d9b640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f7d6c0d3800 0x7f7d6c1573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d69d9b640 1 -- 172.21.15.16:0/1226388059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c162670 msgr2=0x7f7d6c0ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d69d9b640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c162670 0x7f7d6c0ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d69d9b640 1 -- 172.21.15.16:0/1226388059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c0cb280 con 0x7f7d6c15e770 2024-09-12T01:34:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d69d9b640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15e770 0x7f7d6c153af0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400d5f0 tx=0x7f7d5400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d62ffd640 1 -- 172.21.15.16:0/1226388059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d540125f0 con 0x7f7d6c15e770 2024-09-12T01:34:47.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d62ffd640 1 -- 172.21.15.16:0/1226388059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d54019070 con 0x7f7d6c15e770 2024-09-12T01:34:47.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d62ffd640 1 -- 172.21.15.16:0/1226388059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d540129d0 con 0x7f7d6c15e770 2024-09-12T01:34:47.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d709a6640 1 -- 172.21.15.16:0/1226388059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c0cafa0 con 0x7f7d6c15e770 2024-09-12T01:34:47.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.944+0000 7f7d709a6640 1 -- 172.21.15.16:0/1226388059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c154850 con 0x7f7d6c15e770 2024-09-12T01:34:47.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.945+0000 7f7d709a6640 1 -- 172.21.15.16:0/1226388059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d6c154030 con 0x7f7d6c15e770 2024-09-12T01:34:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.946+0000 7f7d62ffd640 1 -- 172.21.15.16:0/1226388059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d54012b70 con 0x7f7d6c15e770 2024-09-12T01:34:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.946+0000 7f7d62ffd640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f7d4c06be30 0x7f7d4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.947+0000 7f7d62ffd640 1 -- 172.21.15.16:0/1226388059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f7d5408afb0 con 0x7f7d6c15e770 2024-09-12T01:34:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.947+0000 7f7d6a59c640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f7d4c06be30 0x7f7d4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.947+0000 7f7d62ffd640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] conn(0x7f7d4c072370 0x7f7d4c0747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.947+0000 7f7d62ffd640 1 -- 172.21.15.16:0/1226388059 --> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d54001d30 con 0x7f7d4c072370 2024-09-12T01:34:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.947+0000 7f7d62ffd640 1 -- 172.21.15.16:0/1226388059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7d54092050 con 0x7f7d6c15e770 2024-09-12T01:34:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.947+0000 7f7d6ad9d640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] conn(0x7f7d4c072370 0x7f7d4c0747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.947+0000 7f7d6a59c640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f7d4c06be30 0x7f7d4c06e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7d64033000 tx=0x7f7d64009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.947+0000 7f7d6ad9d640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] conn(0x7f7d4c072370 0x7f7d4c0747b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.948+0000 7f7d62ffd640 1 -- 172.21.15.16:0/1226388059 <== osd.3 v2:172.21.15.16:6808/1588628599 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d54001d30 con 0x7f7d4c072370 2024-09-12T01:34:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.960+0000 7f7d709a6640 1 -- 172.21.15.16:0/1226388059 --> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7d6c069110 con 0x7f7d4c072370 2024-09-12T01:34:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.961+0000 7f7d62ffd640 1 -- 172.21.15.16:0/1226388059 <== osd.3 v2:172.21.15.16:6808/1588628599 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7d6c069110 con 0x7f7d4c072370 2024-09-12T01:34:47.960 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:34:47.961 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T01:34:47.961 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:34:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.961+0000 7f7d709a6640 1 -- 172.21.15.16:0/1226388059 >> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] conn(0x7f7d4c072370 msgr2=0x7f7d4c0747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.961+0000 7f7d709a6640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.16:6808/1588628599,v1:172.21.15.16:6809/1588628599] conn(0x7f7d4c072370 0x7f7d4c0747b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.961+0000 7f7d709a6640 1 -- 172.21.15.16:0/1226388059 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f7d4c06be30 msgr2=0x7f7d4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.961+0000 7f7d709a6640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f7d4c06be30 0x7f7d4c06e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7d64033000 tx=0x7f7d64009000 comp rx=0 tx=0).stop 2024-09-12T01:34:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.961+0000 7f7d709a6640 1 -- 172.21.15.16:0/1226388059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15e770 msgr2=0x7f7d6c153af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.961+0000 7f7d709a6640 1 --2- 172.21.15.16:0/1226388059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15e770 0x7f7d6c153af0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400d5f0 tx=0x7f7d5400dab0 comp rx=0 tx=0).stop 2024-09-12T01:34:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.961+0000 7f7d6ad9d640 1 -- 172.21.15.16:0/1226388059 reap_dead start 2024-09-12T01:34:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.961+0000 7f7d709a6640 1 -- 172.21.15.16:0/1226388059 shutdown_connections 2024-09-12T01:34:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.961+0000 7f7d709a6640 1 -- 172.21.15.16:0/1226388059 >> 172.21.15.16:0/1226388059 conn(0x7f7d6c09d660 msgr2=0x7f7d6c06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.961+0000 7f7d709a6640 1 -- 172.21.15.16:0/1226388059 shutdown_connections 2024-09-12T01:34:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:47.961+0000 7f7d709a6640 1 -- 172.21.15.16:0/1226388059 wait complete. 2024-09-12T01:34:47.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-12T01:34:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.075+0000 7f64a6a82640 1 Processor -- start 2024-09-12T01:34:48.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.075+0000 7f64a6a82640 1 -- start start 2024-09-12T01:34:48.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.075+0000 7f64a6a82640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64a0147b70 0x7f64a00b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.075+0000 7f64a6a82640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64a015e9f0 0x7f64a0161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.075+0000 7f64a6a82640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f64a006baf0 0x7f64a006bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.075+0000 7f64a6a82640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f64a0162430 con 0x7f64a0147b70 2024-09-12T01:34:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.075+0000 7f64a6a82640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f64a0070330 con 0x7f64a006baf0 2024-09-12T01:34:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.075+0000 7f64a6a82640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f64a0162660 con 0x7f64a015e9f0 2024-09-12T01:34:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.076+0000 7f64a481e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64a0147b70 0x7f64a00b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.076+0000 7f64a481e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64a0147b70 0x7f64a00b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41338/0 (socket says 172.21.15.16:41338) 2024-09-12T01:34:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.076+0000 7f64a481e640 1 -- 172.21.15.16:0/3930721248 learned_addr learned my addr 172.21.15.16:0/3930721248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.076+0000 7f64a501f640 1 --2- 172.21.15.16:0/3930721248 >> v2:172.21.15.33:3300/0 conn(0x7f64a006baf0 0x7f64a006bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.076+0000 7f649ffff640 1 --2- 172.21.15.16:0/3930721248 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64a015e9f0 0x7f64a0161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.076+0000 7f64a481e640 1 -- 172.21.15.16:0/3930721248 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64a015e9f0 msgr2=0x7f64a0161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.076+0000 7f64a481e640 1 --2- 172.21.15.16:0/3930721248 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64a015e9f0 0x7f64a0161ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.076+0000 7f64a481e640 1 -- 172.21.15.16:0/3930721248 >> v2:172.21.15.33:3300/0 conn(0x7f64a006baf0 msgr2=0x7f64a006bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.076+0000 7f64a481e640 1 --2- 172.21.15.16:0/3930721248 >> v2:172.21.15.33:3300/0 conn(0x7f64a006baf0 0x7f64a006bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.076+0000 7f64a481e640 1 -- 172.21.15.16:0/3930721248 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a0162940 con 0x7f64a0147b70 2024-09-12T01:34:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.076+0000 7f649ffff640 1 --2- 172.21.15.16:0/3930721248 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64a015e9f0 0x7f64a0161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.077+0000 7f64a481e640 1 --2- 172.21.15.16:0/3930721248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a0147b70 0x7f64a00b4b20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f64a006c930 tx=0x7f649802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b722a08f282ea106 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.077+0000 7f649f7fe640 1 -- 172.21.15.16:0/3930721248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498036070 con 0x7f64a0147b70 2024-09-12T01:34:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.077+0000 7f649f7fe640 1 -- 172.21.15.16:0/3930721248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6498031600 con 0x7f64a0147b70 2024-09-12T01:34:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.077+0000 7f649f7fe640 1 -- 172.21.15.16:0/3930721248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64980319d0 con 0x7f64a0147b70 2024-09-12T01:34:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.077+0000 7f64a6a82640 1 -- 172.21.15.16:0/3930721248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a0147b70 msgr2=0x7f64a00b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.077+0000 7f64a6a82640 1 --2- 172.21.15.16:0/3930721248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a0147b70 0x7f64a00b4b20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f64a006c930 tx=0x7f649802a6a0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.077+0000 7f64a6a82640 1 -- 172.21.15.16:0/3930721248 shutdown_connections 2024-09-12T01:34:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.077+0000 7f64a6a82640 1 --2- 172.21.15.16:0/3930721248 >> v2:172.21.15.33:3300/0 conn(0x7f64a006baf0 0x7f64a006bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.077+0000 7f64a6a82640 1 --2- 172.21.15.16:0/3930721248 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64a015e9f0 0x7f64a0161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.077+0000 7f64a6a82640 1 --2- 172.21.15.16:0/3930721248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a0147b70 0x7f64a00b4b20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.077+0000 7f64a6a82640 1 -- 172.21.15.16:0/3930721248 >> 172.21.15.16:0/3930721248 conn(0x7f64a00b2e10 msgr2=0x7f64a00b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:48.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.077+0000 7f64a6a82640 1 -- 172.21.15.16:0/3930721248 shutdown_connections 2024-09-12T01:34:48.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.077+0000 7f64a6a82640 1 -- 172.21.15.16:0/3930721248 wait complete. 2024-09-12T01:34:48.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.078+0000 7f64a6a82640 1 Processor -- start 2024-09-12T01:34:48.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.079+0000 7f64a6a82640 1 -- start start 2024-09-12T01:34:48.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.079+0000 7f64a6a82640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64a006baf0 0x7f64a01f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.079+0000 7f64a6a82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a0147b70 0x7f64a00cff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.079+0000 7f64a6a82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015e9f0 0x7f64a0142130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.079+0000 7f64a6a82640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64a00707c0 con 0x7f64a015e9f0 2024-09-12T01:34:48.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.079+0000 7f64a6a82640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0147f50 con 0x7f64a0147b70 2024-09-12T01:34:48.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.079+0000 7f64a6a82640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f64a0070940 con 0x7f64a006baf0 2024-09-12T01:34:48.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f64a501f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015e9f0 0x7f64a0142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f64a481e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64a006baf0 0x7f64a01f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f649ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a0147b70 0x7f64a00cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f649ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a0147b70 0x7f64a00cff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.16:43636/0 (socket says 172.21.15.16:43636) 2024-09-12T01:34:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f64a481e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64a006baf0 0x7f64a01f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3301/0 says I am v2:172.21.15.16:47084/0 (socket says 172.21.15.16:47084) 2024-09-12T01:34:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f64a501f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015e9f0 0x7f64a0142130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41340/0 (socket says 172.21.15.16:41340) 2024-09-12T01:34:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f649ffff640 1 -- 172.21.15.16:0/3789591060 learned_addr learned my addr 172.21.15.16:0/3789591060 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f649ffff640 1 -- 172.21.15.16:0/3789591060 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64a006baf0 msgr2=0x7f64a01f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f649ffff640 1 --2- 172.21.15.16:0/3789591060 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64a006baf0 0x7f64a01f4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f649ffff640 1 -- 172.21.15.16:0/3789591060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015e9f0 msgr2=0x7f64a0142130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f649ffff640 1 --2- 172.21.15.16:0/3789591060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015e9f0 0x7f64a0142130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f649ffff640 1 -- 172.21.15.16:0/3789591060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a00ceec0 con 0x7f64a0147b70 2024-09-12T01:34:48.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f64a501f640 1 --2- 172.21.15.16:0/3789591060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015e9f0 0x7f64a0142130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:48.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f649ffff640 1 --2- 172.21.15.16:0/3789591060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a0147b70 0x7f64a00cff60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f649400d800 tx=0x7f649400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f649d7fa640 1 -- 172.21.15.16:0/3789591060 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940096a0 con 0x7f64a0147b70 2024-09-12T01:34:48.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f64a6a82640 1 -- 172.21.15.16:0/3789591060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a00cf1a0 con 0x7f64a0147b70 2024-09-12T01:34:48.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f649d7fa640 1 -- 172.21.15.16:0/3789591060 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6494019070 con 0x7f64a0147b70 2024-09-12T01:34:48.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f649d7fa640 1 -- 172.21.15.16:0/3789591060 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494009a80 con 0x7f64a0147b70 2024-09-12T01:34:48.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.080+0000 7f64a6a82640 1 -- 172.21.15.16:0/3789591060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a00cf760 con 0x7f64a0147b70 2024-09-12T01:34:48.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.081+0000 7f64a6a82640 1 -- 172.21.15.16:0/3789591060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a00ce4a0 con 0x7f64a0147b70 2024-09-12T01:34:48.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.082+0000 7f649d7fa640 1 -- 172.21.15.16:0/3789591060 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6494009c20 con 0x7f64a0147b70 2024-09-12T01:34:48.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.082+0000 7f649d7fa640 1 --2- 172.21.15.16:0/3789591060 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f648c06be60 0x7f648c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.082+0000 7f649d7fa640 1 -- 172.21.15.16:0/3789591060 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f649408b040 con 0x7f64a0147b70 2024-09-12T01:34:48.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.082+0000 7f64a481e640 1 --2- 172.21.15.16:0/3789591060 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f648c06be60 0x7f648c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.083+0000 7f64a481e640 1 --2- 172.21.15.16:0/3789591060 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f648c06be60 0x7f648c06e320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6498033000 tx=0x7f6498009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.087+0000 7f649d7fa640 1 -- 172.21.15.16:0/3789591060 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6494059420 con 0x7f64a0147b70 2024-09-12T01:34:48.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.264+0000 7f64a6a82640 1 -- 172.21.15.16:0/3789591060 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f64a0069120 con 0x7f648c06be60 2024-09-12T01:34:48.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.264+0000 7f649d7fa640 1 -- 172.21.15.16:0/3789591060 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6498031480 con 0x7f648c06be60 2024-09-12T01:34:48.264 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 3 to deep-scrub 2024-09-12T01:34:48.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.266+0000 7f64a6a82640 1 -- 172.21.15.16:0/3789591060 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f648c06be60 msgr2=0x7f648c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.266+0000 7f64a6a82640 1 --2- 172.21.15.16:0/3789591060 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f648c06be60 0x7f648c06e320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6498033000 tx=0x7f6498009000 comp rx=0 tx=0).stop 2024-09-12T01:34:48.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.266+0000 7f64a6a82640 1 -- 172.21.15.16:0/3789591060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a0147b70 msgr2=0x7f64a00cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.266+0000 7f64a6a82640 1 --2- 172.21.15.16:0/3789591060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a0147b70 0x7f64a00cff60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f649400d800 tx=0x7f649400dcc0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.266+0000 7f64a6a82640 1 -- 172.21.15.16:0/3789591060 shutdown_connections 2024-09-12T01:34:48.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.266+0000 7f64a6a82640 1 --2- 172.21.15.16:0/3789591060 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f648c06be60 0x7f648c06e320 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.266+0000 7f64a6a82640 1 --2- 172.21.15.16:0/3789591060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015e9f0 0x7f64a0142130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.266+0000 7f64a6a82640 1 --2- 172.21.15.16:0/3789591060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64a0147b70 0x7f64a00cff60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.266+0000 7f64a6a82640 1 --2- 172.21.15.16:0/3789591060 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f64a006baf0 0x7f64a01f4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.266+0000 7f64a6a82640 1 -- 172.21.15.16:0/3789591060 >> 172.21.15.16:0/3789591060 conn(0x7f64a00b2e10 msgr2=0x7f64a00744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:48.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.266+0000 7f64a6a82640 1 -- 172.21.15.16:0/3789591060 shutdown_connections 2024-09-12T01:34:48.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.266+0000 7f64a6a82640 1 -- 172.21.15.16:0/3789591060 wait complete. 2024-09-12T01:34:48.277 INFO:tasks.ceph:Scrubbing osd.4 2024-09-12T01:34:48.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-12T01:34:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.384+0000 7fd8eaaba640 1 Processor -- start 2024-09-12T01:34:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.384+0000 7fd8eaaba640 1 -- start start 2024-09-12T01:34:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.385+0000 7fd8eaaba640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd8e415e760 0x7fd8e40d2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.385+0000 7fd8eaaba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8e40cf320 con 0x7fd8e415e380 2024-09-12T01:34:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.385+0000 7fd8eaaba640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8e40d2f30 con 0x7fd8e40d4700 2024-09-12T01:34:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.385+0000 7fd8eaaba640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e40cf620 con 0x7fd8e415e760 2024-09-12T01:34:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.385+0000 7fd8e8856640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd8e415e380 0x7fd8e40d3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48028/0 (socket says 172.21.15.16:48028) 2024-09-12T01:34:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.385+0000 7fd8e8856640 1 -- 172.21.15.16:0/551793197 learned_addr learned my addr 172.21.15.16:0/551793197 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.385+0000 7fd8e3fff640 1 --2- 172.21.15.16:0/551793197 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd8e415e760 0x7fd8e40d2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.385+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 48013917 0 0) 0x7fd8e40cf320 con 0x7fd8e415e380 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.385+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8d4003520 con 0x7fd8e415e380 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.385+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3412119619 0 0) 0x7fd8d4003520 con 0x7fd8e415e380 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.385+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd8e415e760 msgr2=0x7fd8e40d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.385+0000 7fd8e37fe640 1 --2- 172.21.15.16:0/551793197 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd8e415e760 0x7fd8e40d2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.385+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 >> v1:172.21.15.33:6789/0 conn(0x7fd8e40d4700 legacy=0x7fd8e40d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e40d3210 con 0x7fd8e415e380 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7fd8d8001670 con 0x7fd8e415e380 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 >> v1:172.21.15.16:6789/0 conn(0x7fd8e415e380 legacy=0x7fd8e40d3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e37fe640 1 --2- 172.21.15.16:0/551793197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d40039e0 0x7fd8d4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e37fe640 1 --2- 172.21.15.16:0/551793197 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd8e415e760 0x7fd8d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e37fe640 1 --2- 172.21.15.16:0/551793197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8d4004d40 0x7fd8d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e40cf620 con 0x7fd8d4004d40 2024-09-12T01:34:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d400bb50 con 0x7fd8d40039e0 2024-09-12T01:34:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d400bd50 con 0x7fd8e415e760 2024-09-12T01:34:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e8856640 1 --2- 172.21.15.16:0/551793197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d40039e0 0x7fd8d4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e9057640 1 --2- 172.21.15.16:0/551793197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8d4004d40 0x7fd8d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e3fff640 1 --2- 172.21.15.16:0/551793197 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd8e415e760 0x7fd8d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e9057640 1 -- 172.21.15.16:0/551793197 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd8e415e760 msgr2=0x7fd8d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e9057640 1 --2- 172.21.15.16:0/551793197 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd8e415e760 0x7fd8d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e9057640 1 -- 172.21.15.16:0/551793197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d40039e0 msgr2=0x7fd8d4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e9057640 1 --2- 172.21.15.16:0/551793197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d40039e0 0x7fd8d4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e9057640 1 -- 172.21.15.16:0/551793197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d4003520 con 0x7fd8d4004d40 2024-09-12T01:34:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.386+0000 7fd8e8856640 1 --2- 172.21.15.16:0/551793197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d40039e0 0x7fd8d4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.387+0000 7fd8e9057640 1 --2- 172.21.15.16:0/551793197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8d4004d40 0x7fd8d40051c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc001e50 tx=0x7fd8d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad2d974e63c87a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.387+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc001a70 con 0x7fd8d4004d40 2024-09-12T01:34:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.387+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc003970 con 0x7fd8d4004d40 2024-09-12T01:34:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.387+0000 7fd8e37fe640 1 -- 172.21.15.16:0/551793197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc003d60 con 0x7fd8d4004d40 2024-09-12T01:34:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.388+0000 7fd8eaaba640 1 -- 172.21.15.16:0/551793197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8d4004d40 msgr2=0x7fd8d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.388+0000 7fd8eaaba640 1 --2- 172.21.15.16:0/551793197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8d4004d40 0x7fd8d40051c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc001e50 tx=0x7fd8d4005830 comp rx=0 tx=0).stop 2024-09-12T01:34:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.388+0000 7fd8e9057640 1 -- 172.21.15.16:0/551793197 reap_dead start 2024-09-12T01:34:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.388+0000 7fd8eaaba640 1 -- 172.21.15.16:0/551793197 shutdown_connections 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.388+0000 7fd8eaaba640 1 -- 172.21.15.16:0/551793197 >> 172.21.15.16:0/551793197 conn(0x7fd8e408dd10 msgr2=0x7fd8e409d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.388+0000 7fd8eaaba640 1 -- 172.21.15.16:0/551793197 shutdown_connections 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.388+0000 7fd8eaaba640 1 -- 172.21.15.16:0/551793197 wait complete. 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.389+0000 7fd8eaaba640 1 Processor -- start 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.389+0000 7fd8eaaba640 1 -- start start 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8eaaba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8d4004d40 0x7fd8e40b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8eaaba640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd8e40c8360 0x7fd8e40c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8eaaba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8e415ff70 0x7fd8e4160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8eaaba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e40d0c40 con 0x7fd8d4004d40 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8eaaba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e40d0ac0 con 0x7fd8e415ff70 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8eaaba640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e40d0dc0 con 0x7fd8e40c8360 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8e8856640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8d4004d40 0x7fd8e40b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8e8856640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8d4004d40 0x7fd8e40b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41370/0 (socket says 172.21.15.16:41370) 2024-09-12T01:34:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8e8856640 1 -- 172.21.15.16:0/1248425131 learned_addr learned my addr 172.21.15.16:0/1248425131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8e8856640 1 -- 172.21.15.16:0/1248425131 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd8e40c8360 msgr2=0x7fd8e40c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8e8856640 1 --2- 172.21.15.16:0/1248425131 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fd8e40c8360 0x7fd8e40c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8e8856640 1 -- 172.21.15.16:0/1248425131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8e415ff70 msgr2=0x7fd8e4160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8e8856640 1 --2- 172.21.15.16:0/1248425131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8e415ff70 0x7fd8e4160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8e8856640 1 -- 172.21.15.16:0/1248425131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e40b2590 con 0x7fd8d4004d40 2024-09-12T01:34:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.390+0000 7fd8e8856640 1 --2- 172.21.15.16:0/1248425131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8d4004d40 0x7fd8e40b2050 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8007270 tx=0x7fd8d8007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.391+0000 7fd8e17fa640 1 -- 172.21.15.16:0/1248425131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d801a070 con 0x7fd8d4004d40 2024-09-12T01:34:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.391+0000 7fd8e17fa640 1 -- 172.21.15.16:0/1248425131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d800f9a0 con 0x7fd8d4004d40 2024-09-12T01:34:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.391+0000 7fd8e17fa640 1 -- 172.21.15.16:0/1248425131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d800fd90 con 0x7fd8d4004d40 2024-09-12T01:34:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.391+0000 7fd8eaaba640 1 -- 172.21.15.16:0/1248425131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e40b2830 con 0x7fd8d4004d40 2024-09-12T01:34:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.391+0000 7fd8eaaba640 1 -- 172.21.15.16:0/1248425131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e40b2df0 con 0x7fd8d4004d40 2024-09-12T01:34:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.391+0000 7fd8eaaba640 1 -- 172.21.15.16:0/1248425131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd8e40b6ca0 con 0x7fd8d4004d40 2024-09-12T01:34:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.392+0000 7fd8e17fa640 1 -- 172.21.15.16:0/1248425131 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8d801d360 con 0x7fd8d4004d40 2024-09-12T01:34:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.393+0000 7fd8e17fa640 1 --2- 172.21.15.16:0/1248425131 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd8c806be60 0x7fd8c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.393+0000 7fd8e17fa640 1 -- 172.21.15.16:0/1248425131 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7fd8d808a7b0 con 0x7fd8d4004d40 2024-09-12T01:34:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.393+0000 7fd8e3fff640 1 --2- 172.21.15.16:0/1248425131 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd8c806be60 0x7fd8c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.393+0000 7fd8e17fa640 1 --2- 172.21.15.16:0/1248425131 >> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] conn(0x7fd8c80723a0 0x7fd8c80747e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.393+0000 7fd8e17fa640 1 -- 172.21.15.16:0/1248425131 --> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8c8074eb0 con 0x7fd8c80723a0 2024-09-12T01:34:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.393+0000 7fd8e17fa640 1 -- 172.21.15.16:0/1248425131 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd8d808f2c0 con 0x7fd8d4004d40 2024-09-12T01:34:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.393+0000 7fd8e3fff640 1 --2- 172.21.15.16:0/1248425131 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd8c806be60 0x7fd8c806e320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0006a90 tx=0x7fd8d0006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.393+0000 7fd8e9057640 1 --2- 172.21.15.16:0/1248425131 >> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] conn(0x7fd8c80723a0 0x7fd8c80747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.394+0000 7fd8e9057640 1 --2- 172.21.15.16:0/1248425131 >> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] conn(0x7fd8c80723a0 0x7fd8c80747e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.394+0000 7fd8e17fa640 1 -- 172.21.15.16:0/1248425131 <== osd.4 v2:172.21.15.33:6819/1133469969 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd8c8074eb0 con 0x7fd8c80723a0 2024-09-12T01:34:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.407+0000 7fd8eaaba640 1 -- 172.21.15.16:0/1248425131 --> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd8e40b3280 con 0x7fd8c80723a0 2024-09-12T01:34:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.407+0000 7fd8e17fa640 1 -- 172.21.15.16:0/1248425131 <== osd.4 v2:172.21.15.33:6819/1133469969 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd8e40b3280 con 0x7fd8c80723a0 2024-09-12T01:34:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:34:48.407 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T01:34:48.407 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:34:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.408+0000 7fd8eaaba640 1 -- 172.21.15.16:0/1248425131 >> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] conn(0x7fd8c80723a0 msgr2=0x7fd8c80747e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.408+0000 7fd8eaaba640 1 --2- 172.21.15.16:0/1248425131 >> [v2:172.21.15.33:6819/1133469969,v1:172.21.15.33:6821/1133469969] conn(0x7fd8c80723a0 0x7fd8c80747e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.408+0000 7fd8eaaba640 1 -- 172.21.15.16:0/1248425131 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd8c806be60 msgr2=0x7fd8c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.408+0000 7fd8eaaba640 1 --2- 172.21.15.16:0/1248425131 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fd8c806be60 0x7fd8c806e320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0006a90 tx=0x7fd8d0006a20 comp rx=0 tx=0).stop 2024-09-12T01:34:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.408+0000 7fd8eaaba640 1 -- 172.21.15.16:0/1248425131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8d4004d40 msgr2=0x7fd8e40b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.408+0000 7fd8eaaba640 1 --2- 172.21.15.16:0/1248425131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8d4004d40 0x7fd8e40b2050 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8007270 tx=0x7fd8d8007730 comp rx=0 tx=0).stop 2024-09-12T01:34:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.408+0000 7fd8e9057640 1 -- 172.21.15.16:0/1248425131 reap_dead start 2024-09-12T01:34:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.408+0000 7fd8eaaba640 1 -- 172.21.15.16:0/1248425131 shutdown_connections 2024-09-12T01:34:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.408+0000 7fd8eaaba640 1 -- 172.21.15.16:0/1248425131 >> 172.21.15.16:0/1248425131 conn(0x7fd8e408dd10 msgr2=0x7fd8e40b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.408+0000 7fd8eaaba640 1 -- 172.21.15.16:0/1248425131 shutdown_connections 2024-09-12T01:34:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.408+0000 7fd8eaaba640 1 -- 172.21.15.16:0/1248425131 wait complete. 2024-09-12T01:34:48.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-12T01:34:48.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.521+0000 7f321688b640 1 Processor -- start 2024-09-12T01:34:48.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.521+0000 7f321688b640 1 -- start start 2024-09-12T01:34:48.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.522+0000 7f321688b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32101619e0 0x7f32100d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.522+0000 7f321688b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f32100d2d90 0x7f32100d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.522+0000 7f321688b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f32100d1200 0x7f32100b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.522+0000 7f321688b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f32100d3f20 con 0x7f32101619e0 2024-09-12T01:34:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.522+0000 7f321688b640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f32100d16b0 con 0x7f32100d1200 2024-09-12T01:34:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.522+0000 7f321688b640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f3210161dc0 con 0x7f32100d2d90 2024-09-12T01:34:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.522+0000 7f320ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32101619e0 0x7f32100d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.522+0000 7f320ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32101619e0 0x7f32100d2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41376/0 (socket says 172.21.15.16:41376) 2024-09-12T01:34:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.522+0000 7f320ffff640 1 -- 172.21.15.16:0/3506083235 learned_addr learned my addr 172.21.15.16:0/3506083235 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.523+0000 7f3214e28640 1 --2- 172.21.15.16:0/3506083235 >> v2:172.21.15.33:3300/0 conn(0x7f32100d1200 0x7f32100b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.523+0000 7f320ffff640 1 -- 172.21.15.16:0/3506083235 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f32100d2d90 msgr2=0x7f32100d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.523+0000 7f320f7fe640 1 --2- 172.21.15.16:0/3506083235 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f32100d2d90 0x7f32100d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.523+0000 7f320ffff640 1 --2- 172.21.15.16:0/3506083235 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f32100d2d90 0x7f32100d0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.523+0000 7f320ffff640 1 -- 172.21.15.16:0/3506083235 >> v2:172.21.15.33:3300/0 conn(0x7f32100d1200 msgr2=0x7f32100b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.523+0000 7f320ffff640 1 --2- 172.21.15.16:0/3506083235 >> v2:172.21.15.33:3300/0 conn(0x7f32100d1200 0x7f32100b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.523+0000 7f320ffff640 1 -- 172.21.15.16:0/3506083235 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32100d3310 con 0x7f32101619e0 2024-09-12T01:34:48.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.523+0000 7f320ffff640 1 --2- 172.21.15.16:0/3506083235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32101619e0 0x7f32100d2850 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f32100d0ac0 tx=0x7f320002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84aadbaac7ce1269 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.523+0000 7f320effd640 1 -- 172.21.15.16:0/3506083235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200037070 con 0x7f32101619e0 2024-09-12T01:34:48.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.524+0000 7f320effd640 1 -- 172.21.15.16:0/3506083235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3200032440 con 0x7f32101619e0 2024-09-12T01:34:48.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.524+0000 7f320effd640 1 -- 172.21.15.16:0/3506083235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200032810 con 0x7f32101619e0 2024-09-12T01:34:48.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.524+0000 7f321688b640 1 -- 172.21.15.16:0/3506083235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32101619e0 msgr2=0x7f32100d2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.524+0000 7f321688b640 1 --2- 172.21.15.16:0/3506083235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32101619e0 0x7f32100d2850 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f32100d0ac0 tx=0x7f320002a1e0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.524+0000 7f321688b640 1 -- 172.21.15.16:0/3506083235 shutdown_connections 2024-09-12T01:34:48.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.524+0000 7f321688b640 1 --2- 172.21.15.16:0/3506083235 >> v2:172.21.15.33:3300/0 conn(0x7f32100d1200 0x7f32100b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.524+0000 7f321688b640 1 --2- 172.21.15.16:0/3506083235 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f32100d2d90 0x7f32100d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.524+0000 7f321688b640 1 --2- 172.21.15.16:0/3506083235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32101619e0 0x7f32100d2850 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.524+0000 7f321688b640 1 -- 172.21.15.16:0/3506083235 >> 172.21.15.16:0/3506083235 conn(0x7f321006d1c0 msgr2=0x7f3210073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:48.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.524+0000 7f321688b640 1 -- 172.21.15.16:0/3506083235 shutdown_connections 2024-09-12T01:34:48.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.524+0000 7f321688b640 1 -- 172.21.15.16:0/3506083235 wait complete. 2024-09-12T01:34:48.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.525+0000 7f321688b640 1 Processor -- start 2024-09-12T01:34:48.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.526+0000 7f321688b640 1 -- start start 2024-09-12T01:34:48.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f321688b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f32100d1200 0x7f321014f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f321688b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32100d2d90 0x7f3210151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f321688b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32101619e0 0x7f3210152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f321688b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32100d3bb0 con 0x7f32100d2d90 2024-09-12T01:34:48.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f321688b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f32100d38d0 con 0x7f32101619e0 2024-09-12T01:34:48.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f321688b640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f32100d2250 con 0x7f32100d1200 2024-09-12T01:34:48.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f320f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32100d2d90 0x7f3210151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f320f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32100d2d90 0x7f3210151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41388/0 (socket says 172.21.15.16:41388) 2024-09-12T01:34:48.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f320f7fe640 1 -- 172.21.15.16:0/242925186 learned_addr learned my addr 172.21.15.16:0/242925186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:48.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f3214e28640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32101619e0 0x7f3210152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f320ffff640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f32100d1200 0x7f321014f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f320f7fe640 1 -- 172.21.15.16:0/242925186 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f32100d1200 msgr2=0x7f321014f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f320f7fe640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f32100d1200 0x7f321014f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f320f7fe640 1 -- 172.21.15.16:0/242925186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32101619e0 msgr2=0x7f3210152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f320f7fe640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32101619e0 0x7f3210152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f320f7fe640 1 -- 172.21.15.16:0/242925186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32101f5270 con 0x7f32100d2d90 2024-09-12T01:34:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f320f7fe640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32100d2d90 0x7f3210151fd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f320400c820 tx=0x7f320400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f320cff9640 1 -- 172.21.15.16:0/242925186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204015070 con 0x7f32100d2d90 2024-09-12T01:34:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f320cff9640 1 -- 172.21.15.16:0/242925186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3204004220 con 0x7f32100d2d90 2024-09-12T01:34:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f320cff9640 1 -- 172.21.15.16:0/242925186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320400ce10 con 0x7f32100d2d90 2024-09-12T01:34:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.527+0000 7f321688b640 1 -- 172.21.15.16:0/242925186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3210153880 con 0x7f32100d2d90 2024-09-12T01:34:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.528+0000 7f321688b640 1 -- 172.21.15.16:0/242925186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3210153e40 con 0x7f32100d2d90 2024-09-12T01:34:48.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.528+0000 7f321688b640 1 -- 172.21.15.16:0/242925186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32100d0cc0 con 0x7f32100d2d90 2024-09-12T01:34:48.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.529+0000 7f320cff9640 1 -- 172.21.15.16:0/242925186 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3204007830 con 0x7f32100d2d90 2024-09-12T01:34:48.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.529+0000 7f320cff9640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f31fc06be30 0x7f31fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.530+0000 7f320cff9640 1 -- 172.21.15.16:0/242925186 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f320408ac70 con 0x7f32100d2d90 2024-09-12T01:34:48.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.530+0000 7f320ffff640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f31fc06be30 0x7f31fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.530+0000 7f320ffff640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f31fc06be30 0x7f31fc06e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3200034000 tx=0x7f320002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.534+0000 7f320cff9640 1 -- 172.21.15.16:0/242925186 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3204058fc0 con 0x7f32100d2d90 2024-09-12T01:34:48.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.711+0000 7f321688b640 1 -- 172.21.15.16:0/242925186 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f3210158670 con 0x7f31fc06be30 2024-09-12T01:34:48.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.711+0000 7f320cff9640 1 -- 172.21.15.16:0/242925186 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3210158670 con 0x7f31fc06be30 2024-09-12T01:34:48.711 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 4 to deep-scrub 2024-09-12T01:34:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.713+0000 7f321688b640 1 -- 172.21.15.16:0/242925186 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f31fc06be30 msgr2=0x7f31fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.713+0000 7f321688b640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f31fc06be30 0x7f31fc06e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3200034000 tx=0x7f320002e000 comp rx=0 tx=0).stop 2024-09-12T01:34:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.713+0000 7f321688b640 1 -- 172.21.15.16:0/242925186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32100d2d90 msgr2=0x7f3210151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.713+0000 7f321688b640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32100d2d90 0x7f3210151fd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f320400c820 tx=0x7f320400cce0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.713+0000 7f321688b640 1 -- 172.21.15.16:0/242925186 shutdown_connections 2024-09-12T01:34:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.713+0000 7f321688b640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f31fc06be30 0x7f31fc06e2f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.713+0000 7f321688b640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32101619e0 0x7f3210152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.713+0000 7f321688b640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32100d2d90 0x7f3210151fd0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.713+0000 7f321688b640 1 --2- 172.21.15.16:0/242925186 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f32100d1200 0x7f321014f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.713+0000 7f321688b640 1 -- 172.21.15.16:0/242925186 >> 172.21.15.16:0/242925186 conn(0x7f321006d1c0 msgr2=0x7f3210157550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.713+0000 7f321688b640 1 -- 172.21.15.16:0/242925186 shutdown_connections 2024-09-12T01:34:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.713+0000 7f321688b640 1 -- 172.21.15.16:0/242925186 wait complete. 2024-09-12T01:34:48.724 INFO:tasks.ceph:Scrubbing osd.5 2024-09-12T01:34:48.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-12T01:34:48.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.831+0000 7f0f19b9f640 1 Processor -- start 2024-09-12T01:34:48.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.831+0000 7f0f19b9f640 1 -- start start 2024-09-12T01:34:48.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.832+0000 7f0f19b9f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0f1415e380 0x7f0f140d3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.832+0000 7f0f19b9f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f140d2950 0x7f0f140d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.832+0000 7f0f19b9f640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f140cf320 con 0x7f0f1415e760 2024-09-12T01:34:48.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.832+0000 7f0f19b9f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0f140d4a70 con 0x7f0f140d2950 2024-09-12T01:34:48.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.832+0000 7f0f19b9f640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f0f140cf620 con 0x7f0f1415e380 2024-09-12T01:34:48.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.832+0000 7f0f13fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f140d2950 0x7f0f140d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.832+0000 7f0f13fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f140d2950 0x7f0f140d4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41400/0 (socket says 172.21.15.16:41400) 2024-09-12T01:34:48.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.832+0000 7f0f13fff640 1 -- 172.21.15.16:0/3439379186 learned_addr learned my addr 172.21.15.16:0/3439379186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:48.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.832+0000 7f0f137fe640 1 --2- 172.21.15.16:0/3439379186 >> v2:172.21.15.33:3300/0 conn(0x7f0f1415e380 0x7f0f140d3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.832+0000 7f0f13fff640 1 -- 172.21.15.16:0/3439379186 >> v2:172.21.15.33:3300/0 conn(0x7f0f1415e380 msgr2=0x7f0f140d3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:48.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.832+0000 7f0f13fff640 1 --2- 172.21.15.16:0/3439379186 >> v2:172.21.15.33:3300/0 conn(0x7f0f1415e380 0x7f0f140d3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.832+0000 7f0f13fff640 1 -- 172.21.15.16:0/3439379186 >> v1:172.21.15.33:6789/0 conn(0x7f0f1415e760 legacy=0x7f0f140d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:48.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.832+0000 7f0f13fff640 1 -- 172.21.15.16:0/3439379186 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f140d2f30 con 0x7f0f140d2950 2024-09-12T01:34:48.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.833+0000 7f0f13fff640 1 --2- 172.21.15.16:0/3439379186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f140d2950 0x7f0f140d4530 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0f1408c8c0 tx=0x7f0f0c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cdc0aaff12f79d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.833+0000 7f0f127fc640 1 -- 172.21.15.16:0/3439379186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c036070 con 0x7f0f140d2950 2024-09-12T01:34:48.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.833+0000 7f0f127fc640 1 -- 172.21.15.16:0/3439379186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f0c031600 con 0x7f0f140d2950 2024-09-12T01:34:48.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.833+0000 7f0f127fc640 1 -- 172.21.15.16:0/3439379186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c0319d0 con 0x7f0f140d2950 2024-09-12T01:34:48.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.834+0000 7f0f19b9f640 1 -- 172.21.15.16:0/3439379186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f140d2950 msgr2=0x7f0f140d4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.834+0000 7f0f19b9f640 1 --2- 172.21.15.16:0/3439379186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f140d2950 0x7f0f140d4530 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0f1408c8c0 tx=0x7f0f0c02a6f0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.834+0000 7f0f19b9f640 1 -- 172.21.15.16:0/3439379186 shutdown_connections 2024-09-12T01:34:48.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.834+0000 7f0f19b9f640 1 --2- 172.21.15.16:0/3439379186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f140d2950 0x7f0f140d4530 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.834+0000 7f0f19b9f640 1 --2- 172.21.15.16:0/3439379186 >> v2:172.21.15.33:3300/0 conn(0x7f0f1415e380 0x7f0f140d3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.834+0000 7f0f19b9f640 1 -- 172.21.15.16:0/3439379186 >> 172.21.15.16:0/3439379186 conn(0x7f0f1408dd10 msgr2=0x7f0f1409d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:48.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.834+0000 7f0f19b9f640 1 -- 172.21.15.16:0/3439379186 shutdown_connections 2024-09-12T01:34:48.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.834+0000 7f0f19b9f640 1 -- 172.21.15.16:0/3439379186 wait complete. 2024-09-12T01:34:48.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.835+0000 7f0f19b9f640 1 Processor -- start 2024-09-12T01:34:48.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.835+0000 7f0f19b9f640 1 -- start start 2024-09-12T01:34:48.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.836+0000 7f0f19b9f640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f140d2950 0x7f0f1415f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.836+0000 7f0f19b9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1415e380 0x7f0f141614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.836+0000 7f0f19b9f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1415e760 0x7f0f14150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.836+0000 7f0f19b9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f140d3670 con 0x7f0f1415e380 2024-09-12T01:34:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.836+0000 7f0f19b9f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f140d34f0 con 0x7f0f1415e760 2024-09-12T01:34:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.836+0000 7f0f19b9f640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0f140cfab0 con 0x7f0f140d2950 2024-09-12T01:34:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.836+0000 7f0f12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1415e380 0x7f0f141614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.836+0000 7f0f12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1415e380 0x7f0f141614d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41410/0 (socket says 172.21.15.16:41410) 2024-09-12T01:34:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.836+0000 7f0f12ffd640 1 -- 172.21.15.16:0/1978170136 learned_addr learned my addr 172.21.15.16:0/1978170136 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.836+0000 7f0f137fe640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f140d2950 0x7f0f1415f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.836+0000 7f0f13fff640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1415e760 0x7f0f14150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.837+0000 7f0f12ffd640 1 -- 172.21.15.16:0/1978170136 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f140d2950 msgr2=0x7f0f1415f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.837+0000 7f0f12ffd640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0f140d2950 0x7f0f1415f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.837+0000 7f0f12ffd640 1 -- 172.21.15.16:0/1978170136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1415e760 msgr2=0x7f0f14150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.837+0000 7f0f12ffd640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1415e760 0x7f0f14150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.837+0000 7f0f12ffd640 1 -- 172.21.15.16:0/1978170136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f14161a10 con 0x7f0f1415e380 2024-09-12T01:34:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.837+0000 7f0f12ffd640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1415e380 0x7f0f141614d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800b3f0 tx=0x7f0f0800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.837+0000 7f0efbfff640 1 -- 172.21.15.16:0/1978170136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08015020 con 0x7f0f1415e380 2024-09-12T01:34:48.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.837+0000 7f0efbfff640 1 -- 172.21.15.16:0/1978170136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f08011830 con 0x7f0f1415e380 2024-09-12T01:34:48.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.837+0000 7f0f19b9f640 1 -- 172.21.15.16:0/1978170136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f14160020 con 0x7f0f1415e380 2024-09-12T01:34:48.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.837+0000 7f0efbfff640 1 -- 172.21.15.16:0/1978170136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08011c20 con 0x7f0f1415e380 2024-09-12T01:34:48.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.837+0000 7f0f19b9f640 1 -- 172.21.15.16:0/1978170136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f141605e0 con 0x7f0f1415e380 2024-09-12T01:34:48.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.838+0000 7f0f19b9f640 1 -- 172.21.15.16:0/1978170136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f140d2240 con 0x7f0f1415e380 2024-09-12T01:34:48.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.839+0000 7f0efbfff640 1 -- 172.21.15.16:0/1978170136 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f08018020 con 0x7f0f1415e380 2024-09-12T01:34:48.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.839+0000 7f0efbfff640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0f0006be30 0x7f0f0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.839+0000 7f0efbfff640 1 -- 172.21.15.16:0/1978170136 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f0f0808a840 con 0x7f0f1415e380 2024-09-12T01:34:48.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.839+0000 7f0f137fe640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0f0006be30 0x7f0f0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.839+0000 7f0efbfff640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] conn(0x7f0f00072370 0x7f0f000747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.839+0000 7f0efbfff640 1 -- 172.21.15.16:0/1978170136 --> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f0801deb0 con 0x7f0f00072370 2024-09-12T01:34:48.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.839+0000 7f0efbfff640 1 -- 172.21.15.16:0/1978170136 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0f0808f2c0 con 0x7f0f1415e380 2024-09-12T01:34:48.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.839+0000 7f0f13fff640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] conn(0x7f0f00072370 0x7f0f000747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.840+0000 7f0f137fe640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0f0006be30 0x7f0f0006e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0f140d0460 tx=0x7f0f04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.840+0000 7f0f13fff640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] conn(0x7f0f00072370 0x7f0f000747b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.841+0000 7f0efbfff640 1 -- 172.21.15.16:0/1978170136 <== osd.5 v2:172.21.15.33:6808/3986213904 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f0801deb0 con 0x7f0f00072370 2024-09-12T01:34:48.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.853+0000 7f0f19b9f640 1 -- 172.21.15.16:0/1978170136 --> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0f140ce220 con 0x7f0f00072370 2024-09-12T01:34:48.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.854+0000 7f0efbfff640 1 -- 172.21.15.16:0/1978170136 <== osd.5 v2:172.21.15.33:6808/3986213904 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0f140ce220 con 0x7f0f00072370 2024-09-12T01:34:48.854 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:34:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T01:34:48.854 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:34:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.854+0000 7f0f19b9f640 1 -- 172.21.15.16:0/1978170136 >> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] conn(0x7f0f00072370 msgr2=0x7f0f000747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.854+0000 7f0f19b9f640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.33:6808/3986213904,v1:172.21.15.33:6809/3986213904] conn(0x7f0f00072370 0x7f0f000747b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.854+0000 7f0f19b9f640 1 -- 172.21.15.16:0/1978170136 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0f0006be30 msgr2=0x7f0f0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.854+0000 7f0f19b9f640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0f0006be30 0x7f0f0006e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0f140d0460 tx=0x7f0f04008040 comp rx=0 tx=0).stop 2024-09-12T01:34:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.854+0000 7f0f19b9f640 1 -- 172.21.15.16:0/1978170136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1415e380 msgr2=0x7f0f141614d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.854+0000 7f0f19b9f640 1 --2- 172.21.15.16:0/1978170136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1415e380 0x7f0f141614d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800b3f0 tx=0x7f0f0800b8b0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.854+0000 7f0f13fff640 1 -- 172.21.15.16:0/1978170136 reap_dead start 2024-09-12T01:34:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.854+0000 7f0f19b9f640 1 -- 172.21.15.16:0/1978170136 shutdown_connections 2024-09-12T01:34:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.854+0000 7f0f19b9f640 1 -- 172.21.15.16:0/1978170136 >> 172.21.15.16:0/1978170136 conn(0x7f0f1408dd10 msgr2=0x7f0f14159780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.854+0000 7f0f19b9f640 1 -- 172.21.15.16:0/1978170136 shutdown_connections 2024-09-12T01:34:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.855+0000 7f0f19b9f640 1 -- 172.21.15.16:0/1978170136 wait complete. 2024-09-12T01:34:48.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-12T01:34:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.968+0000 7f0e67caf640 1 Processor -- start 2024-09-12T01:34:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.968+0000 7f0e67caf640 1 -- start start 2024-09-12T01:34:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.969+0000 7f0e67caf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0e601619e0 0x7f0e600d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.969+0000 7f0e67caf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e60161dc0 con 0x7f0e600d13d0 2024-09-12T01:34:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.969+0000 7f0e67caf640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e600d3310 con 0x7f0e600d2d90 2024-09-12T01:34:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.969+0000 7f0e67caf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0e600d3f20 con 0x7f0e601619e0 2024-09-12T01:34:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.969+0000 7f0e65a4b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0e601619e0 0x7f0e600d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.969+0000 7f0e6624c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0e600d13d0 0x7f0e600b76c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48042/0 (socket says 172.21.15.16:48042) 2024-09-12T01:34:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.969+0000 7f0e6624c640 1 -- 172.21.15.16:0/337027574 learned_addr learned my addr 172.21.15.16:0/337027574 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.969+0000 7f0e65a4b640 1 -- 172.21.15.16:0/337027574 >> v1:172.21.15.33:6789/0 conn(0x7f0e600d2d90 legacy=0x7f0e600d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.969+0000 7f0e65a4b640 1 -- 172.21.15.16:0/337027574 >> v1:172.21.15.16:6789/0 conn(0x7f0e600d13d0 legacy=0x7f0e600b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.969+0000 7f0e65a4b640 1 -- 172.21.15.16:0/337027574 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e600d35f0 con 0x7f0e601619e0 2024-09-12T01:34:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.969+0000 7f0e64a49640 1 -- 172.21.15.16:0/337027574 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1563270470 0 0) 0x7f0e60161dc0 con 0x7f0e600d13d0 2024-09-12T01:34:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.970+0000 7f0e65a4b640 1 --2- 172.21.15.16:0/337027574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e601619e0 0x7f0e600d2850 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0097a0 tx=0x7f0e5c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2fd2f7bccab4fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.970+0000 7f0e64a49640 1 -- 172.21.15.16:0/337027574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c037070 con 0x7f0e601619e0 2024-09-12T01:34:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.970+0000 7f0e64a49640 1 -- 172.21.15.16:0/337027574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e5c032440 con 0x7f0e601619e0 2024-09-12T01:34:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.970+0000 7f0e64a49640 1 -- 172.21.15.16:0/337027574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c032810 con 0x7f0e601619e0 2024-09-12T01:34:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.971+0000 7f0e67caf640 1 -- 172.21.15.16:0/337027574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e601619e0 msgr2=0x7f0e600d2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.971+0000 7f0e67caf640 1 --2- 172.21.15.16:0/337027574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e601619e0 0x7f0e600d2850 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0097a0 tx=0x7f0e5c02a1e0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.971+0000 7f0e67caf640 1 -- 172.21.15.16:0/337027574 shutdown_connections 2024-09-12T01:34:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.971+0000 7f0e67caf640 1 --2- 172.21.15.16:0/337027574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e601619e0 0x7f0e600d2850 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.971+0000 7f0e67caf640 1 -- 172.21.15.16:0/337027574 >> 172.21.15.16:0/337027574 conn(0x7f0e6006d1c0 msgr2=0x7f0e60073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.971+0000 7f0e67caf640 1 -- 172.21.15.16:0/337027574 shutdown_connections 2024-09-12T01:34:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.971+0000 7f0e67caf640 1 -- 172.21.15.16:0/337027574 wait complete. 2024-09-12T01:34:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.972+0000 7f0e67caf640 1 Processor -- start 2024-09-12T01:34:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.972+0000 7f0e67caf640 1 -- start start 2024-09-12T01:34:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e67caf640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e600d13d0 0x7f0e60152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e67caf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600d2d90 0x7f0e60155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e67caf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e601619e0 0x7f0e60156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e67caf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e600d2250 con 0x7f0e600d2d90 2024-09-12T01:34:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e67caf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e600d3bb0 con 0x7f0e601619e0 2024-09-12T01:34:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e67caf640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f0e600b7dd0 con 0x7f0e600d13d0 2024-09-12T01:34:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e6524a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600d2d90 0x7f0e60155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e6524a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600d2d90 0x7f0e60155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41424/0 (socket says 172.21.15.16:41424) 2024-09-12T01:34:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e6524a640 1 -- 172.21.15.16:0/1901113765 learned_addr learned my addr 172.21.15.16:0/1901113765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e6624c640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e601619e0 0x7f0e60156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e65a4b640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e600d13d0 0x7f0e60152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e6524a640 1 -- 172.21.15.16:0/1901113765 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e600d13d0 msgr2=0x7f0e60152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e6524a640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e600d13d0 0x7f0e60152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e6524a640 1 -- 172.21.15.16:0/1901113765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e601619e0 msgr2=0x7f0e60156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e6524a640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e601619e0 0x7f0e60156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e6524a640 1 -- 172.21.15.16:0/1901113765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e601531f0 con 0x7f0e600d2d90 2024-09-12T01:34:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e65a4b640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e600d13d0 0x7f0e60152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e6624c640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e601619e0 0x7f0e60156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.973+0000 7f0e6524a640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600d2d90 0x7f0e60155b20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000c460 tx=0x7f0e5000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.974+0000 7f0e567fc640 1 -- 172.21.15.16:0/1901113765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50016020 con 0x7f0e600d2d90 2024-09-12T01:34:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.974+0000 7f0e567fc640 1 -- 172.21.15.16:0/1901113765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e50012870 con 0x7f0e600d2d90 2024-09-12T01:34:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.974+0000 7f0e67caf640 1 -- 172.21.15.16:0/1901113765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e60153a70 con 0x7f0e600d2d90 2024-09-12T01:34:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.974+0000 7f0e567fc640 1 -- 172.21.15.16:0/1901113765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50012c60 con 0x7f0e600d2d90 2024-09-12T01:34:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.974+0000 7f0e67caf640 1 -- 172.21.15.16:0/1901113765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e60153d30 con 0x7f0e600d2d90 2024-09-12T01:34:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.975+0000 7f0e67caf640 1 -- 172.21.15.16:0/1901113765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e600cbda0 con 0x7f0e600d2d90 2024-09-12T01:34:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.975+0000 7f0e567fc640 1 -- 172.21.15.16:0/1901113765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e50019020 con 0x7f0e600d2d90 2024-09-12T01:34:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.976+0000 7f0e567fc640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e4406be60 0x7f0e4406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.976+0000 7f0e567fc640 1 -- 172.21.15.16:0/1901113765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f0e5008aaa0 con 0x7f0e600d2d90 2024-09-12T01:34:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.976+0000 7f0e65a4b640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e4406be60 0x7f0e4406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.976+0000 7f0e65a4b640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e4406be60 0x7f0e4406e320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c034000 tx=0x7f0e5c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:48.980+0000 7f0e567fc640 1 -- 172.21.15.16:0/1901113765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0e600cbda0 con 0x7f0e600d2d90 2024-09-12T01:34:49.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.157+0000 7f0e67caf640 1 -- 172.21.15.16:0/1901113765 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f0e600d2850 con 0x7f0e4406be60 2024-09-12T01:34:49.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.157+0000 7f0e567fc640 1 -- 172.21.15.16:0/1901113765 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0e600d2850 con 0x7f0e4406be60 2024-09-12T01:34:49.157 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 5 to deep-scrub 2024-09-12T01:34:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.159+0000 7f0e67caf640 1 -- 172.21.15.16:0/1901113765 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e4406be60 msgr2=0x7f0e4406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.159+0000 7f0e67caf640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e4406be60 0x7f0e4406e320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c034000 tx=0x7f0e5c02e000 comp rx=0 tx=0).stop 2024-09-12T01:34:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.159+0000 7f0e67caf640 1 -- 172.21.15.16:0/1901113765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600d2d90 msgr2=0x7f0e60155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.159+0000 7f0e67caf640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600d2d90 0x7f0e60155b20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000c460 tx=0x7f0e5000c920 comp rx=0 tx=0).stop 2024-09-12T01:34:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.159+0000 7f0e67caf640 1 -- 172.21.15.16:0/1901113765 shutdown_connections 2024-09-12T01:34:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.159+0000 7f0e67caf640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f0e4406be60 0x7f0e4406e320 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.159+0000 7f0e67caf640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e601619e0 0x7f0e60156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.159+0000 7f0e67caf640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600d2d90 0x7f0e60155b20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.159+0000 7f0e67caf640 1 --2- 172.21.15.16:0/1901113765 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f0e600d13d0 0x7f0e60152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.159+0000 7f0e67caf640 1 -- 172.21.15.16:0/1901113765 >> 172.21.15.16:0/1901113765 conn(0x7f0e6006d1c0 msgr2=0x7f0e6015f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.159+0000 7f0e67caf640 1 -- 172.21.15.16:0/1901113765 shutdown_connections 2024-09-12T01:34:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.159+0000 7f0e67caf640 1 -- 172.21.15.16:0/1901113765 wait complete. 2024-09-12T01:34:49.170 INFO:tasks.ceph:Scrubbing osd.6 2024-09-12T01:34:49.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-12T01:34:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.276+0000 7f45f45d6640 1 Processor -- start 2024-09-12T01:34:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.276+0000 7f45f45d6640 1 -- start start 2024-09-12T01:34:49.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.277+0000 7f45f45d6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45ec162130 0x7f45ec162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.277+0000 7f45f45d6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f45ec0b9a80 0x7f45ec0b9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.277+0000 7f45f45d6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45ec0d39d0 con 0x7f45ec15e770 2024-09-12T01:34:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.277+0000 7f45f45d6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f45ec0b7df0 con 0x7f45ec162130 2024-09-12T01:34:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.277+0000 7f45f45d6640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f45ec162ab0 con 0x7f45ec0b9a80 2024-09-12T01:34:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.277+0000 7f45f1b71640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45ec162130 0x7f45ec162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.277+0000 7f45f2372640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f45ec15e770 0x7f45ec0d32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48048/0 (socket says 172.21.15.16:48048) 2024-09-12T01:34:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.277+0000 7f45f2372640 1 -- 172.21.15.16:0/3101929250 learned_addr learned my addr 172.21.15.16:0/3101929250 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.278+0000 7f45f2b73640 1 --2- 172.21.15.16:0/3101929250 >> v2:172.21.15.33:3300/0 conn(0x7f45ec0b9a80 0x7f45ec0b9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.278+0000 7f45f1b71640 1 -- 172.21.15.16:0/3101929250 >> v2:172.21.15.33:3300/0 conn(0x7f45ec0b9a80 msgr2=0x7f45ec0b9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.278+0000 7f45f1b71640 1 --2- 172.21.15.16:0/3101929250 >> v2:172.21.15.33:3300/0 conn(0x7f45ec0b9a80 0x7f45ec0b9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.278+0000 7f45f1b71640 1 -- 172.21.15.16:0/3101929250 >> v1:172.21.15.16:6789/0 conn(0x7f45ec15e770 legacy=0x7f45ec0d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.278+0000 7f45f1370640 1 -- 172.21.15.16:0/3101929250 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1776859278 0 0) 0x7f45ec0d39d0 con 0x7f45ec15e770 2024-09-12T01:34:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.278+0000 7f45f1b71640 1 -- 172.21.15.16:0/3101929250 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec0b74c0 con 0x7f45ec162130 2024-09-12T01:34:49.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.278+0000 7f45f1b71640 1 --2- 172.21.15.16:0/3101929250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45ec162130 0x7f45ec162570 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0cb530 tx=0x7f45e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b797e4694bcccc82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.279+0000 7f45f1370640 1 -- 172.21.15.16:0/3101929250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0037070 con 0x7f45ec162130 2024-09-12T01:34:49.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.279+0000 7f45f1370640 1 -- 172.21.15.16:0/3101929250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e0032440 con 0x7f45ec162130 2024-09-12T01:34:49.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.279+0000 7f45f1370640 1 -- 172.21.15.16:0/3101929250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0032810 con 0x7f45ec162130 2024-09-12T01:34:49.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.279+0000 7f45f45d6640 1 -- 172.21.15.16:0/3101929250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45ec162130 msgr2=0x7f45ec162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.279+0000 7f45f45d6640 1 --2- 172.21.15.16:0/3101929250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45ec162130 0x7f45ec162570 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0cb530 tx=0x7f45e002a1a0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.279+0000 7f45f45d6640 1 -- 172.21.15.16:0/3101929250 shutdown_connections 2024-09-12T01:34:49.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.279+0000 7f45f45d6640 1 --2- 172.21.15.16:0/3101929250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45ec162130 0x7f45ec162570 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.279+0000 7f45f45d6640 1 --2- 172.21.15.16:0/3101929250 >> v2:172.21.15.33:3300/0 conn(0x7f45ec0b9a80 0x7f45ec0b9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.279+0000 7f45f45d6640 1 -- 172.21.15.16:0/3101929250 >> 172.21.15.16:0/3101929250 conn(0x7f45ec09d660 msgr2=0x7f45ec0d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.279+0000 7f45f45d6640 1 -- 172.21.15.16:0/3101929250 shutdown_connections 2024-09-12T01:34:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.279+0000 7f45f45d6640 1 -- 172.21.15.16:0/3101929250 wait complete. 2024-09-12T01:34:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.280+0000 7f45f45d6640 1 Processor -- start 2024-09-12T01:34:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.281+0000 7f45f45d6640 1 -- start start 2024-09-12T01:34:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.281+0000 7f45f45d6640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f45ec0b9a80 0x7f45ec0c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.281+0000 7f45f45d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45ec15e770 0x7f45ec160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.281+0000 7f45f45d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45ec162130 0x7f45ec1f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.281+0000 7f45f45d6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec0d51c0 con 0x7f45ec15e770 2024-09-12T01:34:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.281+0000 7f45f45d6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec0b7a80 con 0x7f45ec162130 2024-09-12T01:34:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.281+0000 7f45f45d6640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f45ec0d5340 con 0x7f45ec0b9a80 2024-09-12T01:34:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.281+0000 7f45f1b71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45ec15e770 0x7f45ec160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.281+0000 7f45f1b71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45ec15e770 0x7f45ec160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41448/0 (socket says 172.21.15.16:41448) 2024-09-12T01:34:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.281+0000 7f45f1b71640 1 -- 172.21.15.16:0/1705044609 learned_addr learned my addr 172.21.15.16:0/1705044609 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.281+0000 7f45f2b73640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45ec162130 0x7f45ec1f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.282+0000 7f45f2372640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f45ec0b9a80 0x7f45ec0c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.282+0000 7f45f1b71640 1 -- 172.21.15.16:0/1705044609 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f45ec0b9a80 msgr2=0x7f45ec0c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.282+0000 7f45f1b71640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f45ec0b9a80 0x7f45ec0c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.282+0000 7f45f1b71640 1 -- 172.21.15.16:0/1705044609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45ec162130 msgr2=0x7f45ec1f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.282+0000 7f45f1b71640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45ec162130 0x7f45ec1f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.282+0000 7f45f1b71640 1 -- 172.21.15.16:0/1705044609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec15f720 con 0x7f45ec15e770 2024-09-12T01:34:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.282+0000 7f45f2372640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f45ec0b9a80 0x7f45ec0c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.282+0000 7f45f1b71640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45ec15e770 0x7f45ec160ba0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009890 tx=0x7f45e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.282+0000 7f45eaffd640 1 -- 172.21.15.16:0/1705044609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0037040 con 0x7f45ec15e770 2024-09-12T01:34:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.282+0000 7f45eaffd640 1 -- 172.21.15.16:0/1705044609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e0042070 con 0x7f45ec15e770 2024-09-12T01:34:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.282+0000 7f45f45d6640 1 -- 172.21.15.16:0/1705044609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ec15fa00 con 0x7f45ec15e770 2024-09-12T01:34:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.282+0000 7f45eaffd640 1 -- 172.21.15.16:0/1705044609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0030bd0 con 0x7f45ec15e770 2024-09-12T01:34:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.282+0000 7f45f45d6640 1 -- 172.21.15.16:0/1705044609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ec1602a0 con 0x7f45ec15e770 2024-09-12T01:34:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.283+0000 7f45f45d6640 1 -- 172.21.15.16:0/1705044609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45ec069110 con 0x7f45ec15e770 2024-09-12T01:34:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.284+0000 7f45eaffd640 1 -- 172.21.15.16:0/1705044609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45e0047460 con 0x7f45ec15e770 2024-09-12T01:34:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.284+0000 7f45eaffd640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f45d806be30 0x7f45d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.284+0000 7f45eaffd640 1 -- 172.21.15.16:0/1705044609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f45e00ab880 con 0x7f45ec15e770 2024-09-12T01:34:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.284+0000 7f45f2372640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f45d806be30 0x7f45d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.284+0000 7f45eaffd640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] conn(0x7f45d8072370 0x7f45d80747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.284+0000 7f45eaffd640 1 -- 172.21.15.16:0/1705044609 --> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45e0008eb0 con 0x7f45d8072370 2024-09-12T01:34:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.284+0000 7f45eaffd640 1 -- 172.21.15.16:0/1705044609 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f45e00b02c0 con 0x7f45ec15e770 2024-09-12T01:34:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.284+0000 7f45f2b73640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] conn(0x7f45d8072370 0x7f45d80747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.285+0000 7f45f2372640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f45d806be30 0x7f45d806e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f45e4000f60 tx=0x7f45e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.285+0000 7f45f2b73640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] conn(0x7f45d8072370 0x7f45d80747b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.286+0000 7f45eaffd640 1 -- 172.21.15.16:0/1705044609 <== osd.6 v2:172.21.15.33:6816/1471795647 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45e0008eb0 con 0x7f45d8072370 2024-09-12T01:34:49.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.298+0000 7f45f45d6640 1 -- 172.21.15.16:0/1705044609 --> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f45ec162570 con 0x7f45d8072370 2024-09-12T01:34:49.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.299+0000 7f45eaffd640 1 -- 172.21.15.16:0/1705044609 <== osd.6 v2:172.21.15.33:6816/1471795647 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f45ec162570 con 0x7f45d8072370 2024-09-12T01:34:49.299 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:34:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T01:34:49.299 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:34:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.299+0000 7f45f45d6640 1 -- 172.21.15.16:0/1705044609 >> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] conn(0x7f45d8072370 msgr2=0x7f45d80747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.299+0000 7f45f45d6640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.33:6816/1471795647,v1:172.21.15.33:6817/1471795647] conn(0x7f45d8072370 0x7f45d80747b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.299+0000 7f45f45d6640 1 -- 172.21.15.16:0/1705044609 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f45d806be30 msgr2=0x7f45d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.299+0000 7f45f45d6640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f45d806be30 0x7f45d806e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f45e4000f60 tx=0x7f45e4009040 comp rx=0 tx=0).stop 2024-09-12T01:34:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.299+0000 7f45f45d6640 1 -- 172.21.15.16:0/1705044609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45ec15e770 msgr2=0x7f45ec160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.299+0000 7f45f45d6640 1 --2- 172.21.15.16:0/1705044609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45ec15e770 0x7f45ec160ba0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009890 tx=0x7f45e00046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.299+0000 7f45f2b73640 1 -- 172.21.15.16:0/1705044609 reap_dead start 2024-09-12T01:34:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.299+0000 7f45f45d6640 1 -- 172.21.15.16:0/1705044609 shutdown_connections 2024-09-12T01:34:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.299+0000 7f45f45d6640 1 -- 172.21.15.16:0/1705044609 >> 172.21.15.16:0/1705044609 conn(0x7f45ec09d660 msgr2=0x7f45ec06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.299+0000 7f45f45d6640 1 -- 172.21.15.16:0/1705044609 shutdown_connections 2024-09-12T01:34:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.299+0000 7f45f45d6640 1 -- 172.21.15.16:0/1705044609 wait complete. 2024-09-12T01:34:49.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-12T01:34:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.413+0000 7f94a57c6640 1 Processor -- start 2024-09-12T01:34:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.413+0000 7f94a57c6640 1 -- start start 2024-09-12T01:34:49.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.414+0000 7f94a57c6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f94a00d4d60 0x7f94a00d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.414+0000 7f94a57c6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a0162500 con 0x7f94a0162020 2024-09-12T01:34:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.414+0000 7f94a57c6640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a01627e0 con 0x7f94a015e660 2024-09-12T01:34:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.414+0000 7f94a57c6640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f94a00d2020 con 0x7f94a00d4d60 2024-09-12T01:34:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.414+0000 7f949f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f94a0162020 0x7f94a00d29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48050/0 (socket says 172.21.15.16:48050) 2024-09-12T01:34:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.414+0000 7f949f7fe640 1 -- 172.21.15.16:0/3425772127 learned_addr learned my addr 172.21.15.16:0/3425772127 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.415+0000 7f949e7fc640 1 --2- 172.21.15.16:0/3425772127 >> v2:172.21.15.33:3300/0 conn(0x7f94a00d4d60 0x7f94a00d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.415+0000 7f949dffb640 1 -- 172.21.15.16:0/3425772127 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1604753805 0 0) 0x7f94a0162500 con 0x7f94a0162020 2024-09-12T01:34:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.415+0000 7f949dffb640 1 -- 172.21.15.16:0/3425772127 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f948c003520 con 0x7f94a0162020 2024-09-12T01:34:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.415+0000 7f949dffb640 1 -- 172.21.15.16:0/3425772127 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2822736800 0 0) 0x7f94a01627e0 con 0x7f94a015e660 2024-09-12T01:34:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.415+0000 7f949dffb640 1 -- 172.21.15.16:0/3425772127 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94a0162500 con 0x7f94a015e660 2024-09-12T01:34:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.415+0000 7f949e7fc640 1 -- 172.21.15.16:0/3425772127 >> v1:172.21.15.33:6789/0 conn(0x7f94a015e660 legacy=0x7f94a00d4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.415+0000 7f949e7fc640 1 -- 172.21.15.16:0/3425772127 >> v1:172.21.15.16:6789/0 conn(0x7f94a0162020 legacy=0x7f94a00d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.415+0000 7f949dffb640 1 -- 172.21.15.16:0/3425772127 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3125921098 0 0) 0x7f948c003520 con 0x7f94a0162020 2024-09-12T01:34:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.415+0000 7f949e7fc640 1 -- 172.21.15.16:0/3425772127 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a0162ac0 con 0x7f94a00d4d60 2024-09-12T01:34:49.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.416+0000 7f949e7fc640 1 --2- 172.21.15.16:0/3425772127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a00d4d60 0x7f94a00d24b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f94880097a0 tx=0x7f948802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ea92b5fee54895 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.416+0000 7f949dffb640 1 -- 172.21.15.16:0/3425772127 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488037070 con 0x7f94a00d4d60 2024-09-12T01:34:49.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.416+0000 7f949dffb640 1 -- 172.21.15.16:0/3425772127 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f948802abc0 con 0x7f94a00d4d60 2024-09-12T01:34:49.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.416+0000 7f949dffb640 1 -- 172.21.15.16:0/3425772127 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498001670 con 0x7f94a00d4d60 2024-09-12T01:34:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.416+0000 7f94a57c6640 1 -- 172.21.15.16:0/3425772127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a00d4d60 msgr2=0x7f94a00d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.416+0000 7f94a57c6640 1 --2- 172.21.15.16:0/3425772127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a00d4d60 0x7f94a00d24b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f94880097a0 tx=0x7f948802a130 comp rx=0 tx=0).stop 2024-09-12T01:34:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.417+0000 7f94a57c6640 1 -- 172.21.15.16:0/3425772127 shutdown_connections 2024-09-12T01:34:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.417+0000 7f94a57c6640 1 --2- 172.21.15.16:0/3425772127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a00d4d60 0x7f94a00d24b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.417+0000 7f94a57c6640 1 -- 172.21.15.16:0/3425772127 >> 172.21.15.16:0/3425772127 conn(0x7f94a008de10 msgr2=0x7f94a009d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.417+0000 7f94a57c6640 1 -- 172.21.15.16:0/3425772127 shutdown_connections 2024-09-12T01:34:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.417+0000 7f94a57c6640 1 -- 172.21.15.16:0/3425772127 wait complete. 2024-09-12T01:34:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.418+0000 7f94a57c6640 1 Processor -- start 2024-09-12T01:34:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.418+0000 7f94a57c6640 1 -- start start 2024-09-12T01:34:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f94a57c6640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f94a00d4d60 0x7f94a01599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f94a57c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a015e660 0x7f94a00c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f94a57c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a0162020 0x7f94a00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f94a57c6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f94a00d4990 con 0x7f94a015e660 2024-09-12T01:34:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f94a57c6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94a00d4810 con 0x7f94a0162020 2024-09-12T01:34:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f94a57c6640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f94a00d4b10 con 0x7f94a00d4d60 2024-09-12T01:34:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f949e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a015e660 0x7f94a00c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f949e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a015e660 0x7f94a00c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41460/0 (socket says 172.21.15.16:41460) 2024-09-12T01:34:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f949e7fc640 1 -- 172.21.15.16:0/3219855451 learned_addr learned my addr 172.21.15.16:0/3219855451 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f949f7fe640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a0162020 0x7f94a00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f949effd640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f94a00d4d60 0x7f94a01599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f949e7fc640 1 -- 172.21.15.16:0/3219855451 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f94a00d4d60 msgr2=0x7f94a01599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f949e7fc640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f94a00d4d60 0x7f94a01599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f949e7fc640 1 -- 172.21.15.16:0/3219855451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a0162020 msgr2=0x7f94a00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f949e7fc640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a0162020 0x7f94a00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f949e7fc640 1 -- 172.21.15.16:0/3219855451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a0159ef0 con 0x7f94a015e660 2024-09-12T01:34:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f949f7fe640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a0162020 0x7f94a00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f949effd640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f94a00d4d60 0x7f94a01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f949e7fc640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a015e660 0x7f94a00c9230 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9488035040 tx=0x7f94880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f94977fe640 1 -- 172.21.15.16:0/3219855451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488037050 con 0x7f94a015e660 2024-09-12T01:34:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f94977fe640 1 -- 172.21.15.16:0/3219855451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9488043070 con 0x7f94a015e660 2024-09-12T01:34:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.419+0000 7f94a57c6640 1 -- 172.21.15.16:0/3219855451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a00b9670 con 0x7f94a015e660 2024-09-12T01:34:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.420+0000 7f94977fe640 1 -- 172.21.15.16:0/3219855451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488006e10 con 0x7f94a015e660 2024-09-12T01:34:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.420+0000 7f94a57c6640 1 -- 172.21.15.16:0/3219855451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a00b9c30 con 0x7f94a015e660 2024-09-12T01:34:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.421+0000 7f94a57c6640 1 -- 172.21.15.16:0/3219855451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a00d0f50 con 0x7f94a015e660 2024-09-12T01:34:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.421+0000 7f94977fe640 1 -- 172.21.15.16:0/3219855451 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9488048460 con 0x7f94a015e660 2024-09-12T01:34:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.422+0000 7f94977fe640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f948006be30 0x7f948006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.422+0000 7f949effd640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f948006be30 0x7f948006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.422+0000 7f94977fe640 1 -- 172.21.15.16:0/3219855451 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f94880ab4e0 con 0x7f94a015e660 2024-09-12T01:34:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.422+0000 7f949effd640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f948006be30 0x7f948006e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9490007fd0 tx=0x7f9490009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.426+0000 7f94977fe640 1 -- 172.21.15.16:0/3219855451 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f94a0159ef0 con 0x7f94a015e660 2024-09-12T01:34:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.602+0000 7f94a57c6640 1 -- 172.21.15.16:0/3219855451 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f94a00b4090 con 0x7f948006be30 2024-09-12T01:34:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.602+0000 7f94977fe640 1 -- 172.21.15.16:0/3219855451 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f94a00b4090 con 0x7f948006be30 2024-09-12T01:34:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 6 to deep-scrub 2024-09-12T01:34:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.604+0000 7f94a57c6640 1 -- 172.21.15.16:0/3219855451 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f948006be30 msgr2=0x7f948006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.604+0000 7f94a57c6640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f948006be30 0x7f948006e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9490007fd0 tx=0x7f9490009040 comp rx=0 tx=0).stop 2024-09-12T01:34:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.604+0000 7f94a57c6640 1 -- 172.21.15.16:0/3219855451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a015e660 msgr2=0x7f94a00c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.604+0000 7f94a57c6640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a015e660 0x7f94a00c9230 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9488035040 tx=0x7f94880046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.604+0000 7f94a57c6640 1 -- 172.21.15.16:0/3219855451 shutdown_connections 2024-09-12T01:34:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.604+0000 7f94a57c6640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f948006be30 0x7f948006e2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.604+0000 7f94a57c6640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a0162020 0x7f94a00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.604+0000 7f94a57c6640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a015e660 0x7f94a00c9230 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.604+0000 7f94a57c6640 1 --2- 172.21.15.16:0/3219855451 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f94a00d4d60 0x7f94a01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.604+0000 7f94a57c6640 1 -- 172.21.15.16:0/3219855451 >> 172.21.15.16:0/3219855451 conn(0x7f94a008de10 msgr2=0x7f94a006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:49.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.604+0000 7f94a57c6640 1 -- 172.21.15.16:0/3219855451 shutdown_connections 2024-09-12T01:34:49.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.604+0000 7f94a57c6640 1 -- 172.21.15.16:0/3219855451 wait complete. 2024-09-12T01:34:49.615 INFO:tasks.ceph:Scrubbing osd.7 2024-09-12T01:34:49.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-12T01:34:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.722+0000 7feb6e465640 1 Processor -- start 2024-09-12T01:34:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.722+0000 7feb6e465640 1 -- start start 2024-09-12T01:34:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.723+0000 7feb6e465640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb680d3050 0x7feb680d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.723+0000 7feb6e465640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb680d11d0 0x7feb680b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.723+0000 7feb6e465640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb680d15e0 con 0x7feb6815e4d0 2024-09-12T01:34:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.723+0000 7feb6e465640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7feb680d1fb0 con 0x7feb680d3050 2024-09-12T01:34:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.723+0000 7feb6e465640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7feb680d2290 con 0x7feb680d11d0 2024-09-12T01:34:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.723+0000 7feb677fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb680d3050 0x7feb680d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.723+0000 7feb677fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb680d3050 0x7feb680d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41462/0 (socket says 172.21.15.16:41462) 2024-09-12T01:34:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.723+0000 7feb677fe640 1 -- 172.21.15.16:0/16736275 learned_addr learned my addr 172.21.15.16:0/16736275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.723+0000 7feb6ca02640 1 --2- 172.21.15.16:0/16736275 >> v2:172.21.15.33:3300/0 conn(0x7feb680d11d0 0x7feb680b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.723+0000 7feb677fe640 1 -- 172.21.15.16:0/16736275 >> v2:172.21.15.33:3300/0 conn(0x7feb680d11d0 msgr2=0x7feb680b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.723+0000 7feb677fe640 1 --2- 172.21.15.16:0/16736275 >> v2:172.21.15.33:3300/0 conn(0x7feb680d11d0 0x7feb680b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.723+0000 7feb677fe640 1 -- 172.21.15.16:0/16736275 >> v1:172.21.15.33:6789/0 conn(0x7feb6815e4d0 legacy=0x7feb680cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.723+0000 7feb677fe640 1 -- 172.21.15.16:0/16736275 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb680d2570 con 0x7feb680d3050 2024-09-12T01:34:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.724+0000 7feb677fe640 1 --2- 172.21.15.16:0/16736275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb680d3050 0x7feb680d0c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feb680d3650 tx=0x7feb5c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90e7995466a2543 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.724+0000 7feb66ffd640 1 -- 172.21.15.16:0/16736275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c037070 con 0x7feb680d3050 2024-09-12T01:34:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.724+0000 7feb66ffd640 1 -- 172.21.15.16:0/16736275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5c032440 con 0x7feb680d3050 2024-09-12T01:34:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.724+0000 7feb66ffd640 1 -- 172.21.15.16:0/16736275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c032810 con 0x7feb680d3050 2024-09-12T01:34:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.725+0000 7feb6e465640 1 -- 172.21.15.16:0/16736275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb680d3050 msgr2=0x7feb680d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.725+0000 7feb6e465640 1 --2- 172.21.15.16:0/16736275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb680d3050 0x7feb680d0c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feb680d3650 tx=0x7feb5c02a1a0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.725+0000 7feb6e465640 1 -- 172.21.15.16:0/16736275 shutdown_connections 2024-09-12T01:34:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.725+0000 7feb6e465640 1 --2- 172.21.15.16:0/16736275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb680d3050 0x7feb680d0c90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.725+0000 7feb6e465640 1 --2- 172.21.15.16:0/16736275 >> v2:172.21.15.33:3300/0 conn(0x7feb680d11d0 0x7feb680b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.725+0000 7feb6e465640 1 -- 172.21.15.16:0/16736275 >> 172.21.15.16:0/16736275 conn(0x7feb6809d650 msgr2=0x7feb680d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.725+0000 7feb6e465640 1 -- 172.21.15.16:0/16736275 shutdown_connections 2024-09-12T01:34:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.725+0000 7feb6e465640 1 -- 172.21.15.16:0/16736275 wait complete. 2024-09-12T01:34:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.726+0000 7feb6e465640 1 Processor -- start 2024-09-12T01:34:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.726+0000 7feb6e465640 1 -- start start 2024-09-12T01:34:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6e465640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb680d11d0 0x7feb680b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6e465640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb680d3050 0x7feb680b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6e465640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb6815e4d0 0x7feb680b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6e465640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb680d4d80 con 0x7feb6815e4d0 2024-09-12T01:34:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6e465640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb680d2b30 con 0x7feb680d11d0 2024-09-12T01:34:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6e465640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7feb680d01a0 con 0x7feb680d3050 2024-09-12T01:34:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6ca02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb6815e4d0 0x7feb680b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6ca02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb6815e4d0 0x7feb680b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41472/0 (socket says 172.21.15.16:41472) 2024-09-12T01:34:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6ca02640 1 -- 172.21.15.16:0/1520664018 learned_addr learned my addr 172.21.15.16:0/1520664018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb67fff640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb680d11d0 0x7feb680b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb677fe640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb680d3050 0x7feb680b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6ca02640 1 -- 172.21.15.16:0/1520664018 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb680d3050 msgr2=0x7feb680b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6ca02640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feb680d3050 0x7feb680b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6ca02640 1 -- 172.21.15.16:0/1520664018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb680d11d0 msgr2=0x7feb680b98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6ca02640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb680d11d0 0x7feb680b98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.727+0000 7feb6ca02640 1 -- 172.21.15.16:0/1520664018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb680cc9f0 con 0x7feb6815e4d0 2024-09-12T01:34:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.728+0000 7feb6ca02640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb6815e4d0 0x7feb680b18d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feb60007e40 tx=0x7feb6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.728+0000 7feb64ff9640 1 -- 172.21.15.16:0/1520664018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60014620 con 0x7feb6815e4d0 2024-09-12T01:34:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.728+0000 7feb64ff9640 1 -- 172.21.15.16:0/1520664018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb60019070 con 0x7feb6815e4d0 2024-09-12T01:34:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.728+0000 7feb6e465640 1 -- 172.21.15.16:0/1520664018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb68156a30 con 0x7feb6815e4d0 2024-09-12T01:34:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.728+0000 7feb64ff9640 1 -- 172.21.15.16:0/1520664018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60014a20 con 0x7feb6815e4d0 2024-09-12T01:34:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.728+0000 7feb6e465640 1 -- 172.21.15.16:0/1520664018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb680b7ae0 con 0x7feb6815e4d0 2024-09-12T01:34:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.729+0000 7feb6e465640 1 -- 172.21.15.16:0/1520664018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb68069110 con 0x7feb6815e4d0 2024-09-12T01:34:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.729+0000 7feb64ff9640 1 -- 172.21.15.16:0/1520664018 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb60014bc0 con 0x7feb6815e4d0 2024-09-12T01:34:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.730+0000 7feb64ff9640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feb5406be60 0x7feb5406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.730+0000 7feb64ff9640 1 -- 172.21.15.16:0/1520664018 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7feb6008b4b0 con 0x7feb6815e4d0 2024-09-12T01:34:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.730+0000 7feb67fff640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feb5406be60 0x7feb5406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.730+0000 7feb64ff9640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] conn(0x7feb540723a0 0x7feb540747e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.730+0000 7feb64ff9640 1 -- 172.21.15.16:0/1520664018 --> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb6000df30 con 0x7feb540723a0 2024-09-12T01:34:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.730+0000 7feb64ff9640 1 -- 172.21.15.16:0/1520664018 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7feb60092050 con 0x7feb6815e4d0 2024-09-12T01:34:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.730+0000 7feb67fff640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feb5406be60 0x7feb5406e320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feb680b97b0 tx=0x7feb58009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.730+0000 7feb677fe640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] conn(0x7feb540723a0 0x7feb540747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.731+0000 7feb677fe640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] conn(0x7feb540723a0 0x7feb540747e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.731+0000 7feb64ff9640 1 -- 172.21.15.16:0/1520664018 <== osd.7 v2:172.21.15.33:6800/2262537058 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb6000df30 con 0x7feb540723a0 2024-09-12T01:34:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.744+0000 7feb6e465640 1 -- 172.21.15.16:0/1520664018 --> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7feb6815e8b0 con 0x7feb540723a0 2024-09-12T01:34:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.745+0000 7feb64ff9640 1 -- 172.21.15.16:0/1520664018 <== osd.7 v2:172.21.15.33:6800/2262537058 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7feb6815e8b0 con 0x7feb540723a0 2024-09-12T01:34:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:34:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T01:34:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:34:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.745+0000 7feb6e465640 1 -- 172.21.15.16:0/1520664018 >> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] conn(0x7feb540723a0 msgr2=0x7feb540747e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.745+0000 7feb6e465640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.33:6800/2262537058,v1:172.21.15.33:6801/2262537058] conn(0x7feb540723a0 0x7feb540747e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.745+0000 7feb6e465640 1 -- 172.21.15.16:0/1520664018 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feb5406be60 msgr2=0x7feb5406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.745+0000 7feb6e465640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7feb5406be60 0x7feb5406e320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feb680b97b0 tx=0x7feb58009060 comp rx=0 tx=0).stop 2024-09-12T01:34:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.745+0000 7feb6e465640 1 -- 172.21.15.16:0/1520664018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb6815e4d0 msgr2=0x7feb680b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.745+0000 7feb6e465640 1 --2- 172.21.15.16:0/1520664018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb6815e4d0 0x7feb680b18d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feb60007e40 tx=0x7feb6000e4b0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.745+0000 7feb6ca02640 1 -- 172.21.15.16:0/1520664018 reap_dead start 2024-09-12T01:34:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.745+0000 7feb6e465640 1 -- 172.21.15.16:0/1520664018 shutdown_connections 2024-09-12T01:34:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.745+0000 7feb6e465640 1 -- 172.21.15.16:0/1520664018 >> 172.21.15.16:0/1520664018 conn(0x7feb6809d650 msgr2=0x7feb680b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.745+0000 7feb6e465640 1 -- 172.21.15.16:0/1520664018 shutdown_connections 2024-09-12T01:34:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.745+0000 7feb6e465640 1 -- 172.21.15.16:0/1520664018 wait complete. 2024-09-12T01:34:49.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-12T01:34:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.860+0000 7f19fb035640 1 Processor -- start 2024-09-12T01:34:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.860+0000 7f19fb035640 1 -- start start 2024-09-12T01:34:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.860+0000 7f19fb035640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19f415e400 0x7f19f4072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.860+0000 7f19fb035640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f19f40ceb70 0x7f19f406cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.861+0000 7f19fb035640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f4072650 con 0x7f19f406d140 2024-09-12T01:34:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.861+0000 7f19fb035640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f19f40cf130 con 0x7f19f415e400 2024-09-12T01:34:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.861+0000 7f19fb035640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f19f40cf410 con 0x7f19f40ceb70 2024-09-12T01:34:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.861+0000 7f19f8dd1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19f415e400 0x7f19f4072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.861+0000 7f19f95d2640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f19f406d140 0x7f19f415f010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48052/0 (socket says 172.21.15.16:48052) 2024-09-12T01:34:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.861+0000 7f19f95d2640 1 -- 172.21.15.16:0/957568478 learned_addr learned my addr 172.21.15.16:0/957568478 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.861+0000 7f19f3fff640 1 --2- 172.21.15.16:0/957568478 >> v2:172.21.15.33:3300/0 conn(0x7f19f40ceb70 0x7f19f406cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.861+0000 7f19f8dd1640 1 -- 172.21.15.16:0/957568478 >> v2:172.21.15.33:3300/0 conn(0x7f19f40ceb70 msgr2=0x7f19f406cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.861+0000 7f19f8dd1640 1 --2- 172.21.15.16:0/957568478 >> v2:172.21.15.33:3300/0 conn(0x7f19f40ceb70 0x7f19f406cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.861+0000 7f19f8dd1640 1 -- 172.21.15.16:0/957568478 >> v1:172.21.15.16:6789/0 conn(0x7f19f406d140 legacy=0x7f19f415f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.861+0000 7f19f8dd1640 1 -- 172.21.15.16:0/957568478 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f40cf6f0 con 0x7f19f415e400 2024-09-12T01:34:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.861+0000 7f19f37fe640 1 -- 172.21.15.16:0/957568478 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 452219716 0 0) 0x7f19f4072650 con 0x7f19f406d140 2024-09-12T01:34:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.862+0000 7f19f8dd1640 1 --2- 172.21.15.16:0/957568478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f415e400 0x7f19f4072110 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f19f40d3030 tx=0x7f19e802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f695add7df60b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.862+0000 7f19f37fe640 1 -- 172.21.15.16:0/957568478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8037070 con 0x7f19f415e400 2024-09-12T01:34:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.862+0000 7f19f37fe640 1 -- 172.21.15.16:0/957568478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8032440 con 0x7f19f415e400 2024-09-12T01:34:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.862+0000 7f19f37fe640 1 -- 172.21.15.16:0/957568478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8032810 con 0x7f19f415e400 2024-09-12T01:34:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.862+0000 7f19fb035640 1 -- 172.21.15.16:0/957568478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f415e400 msgr2=0x7f19f4072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.862+0000 7f19fb035640 1 --2- 172.21.15.16:0/957568478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f415e400 0x7f19f4072110 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f19f40d3030 tx=0x7f19e802a1a0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.863+0000 7f19fb035640 1 -- 172.21.15.16:0/957568478 shutdown_connections 2024-09-12T01:34:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.863+0000 7f19fb035640 1 --2- 172.21.15.16:0/957568478 >> v2:172.21.15.33:3300/0 conn(0x7f19f40ceb70 0x7f19f406cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.863+0000 7f19fb035640 1 --2- 172.21.15.16:0/957568478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f415e400 0x7f19f4072110 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.863+0000 7f19fb035640 1 -- 172.21.15.16:0/957568478 >> 172.21.15.16:0/957568478 conn(0x7f19f406feb0 msgr2=0x7f19f4070290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.863+0000 7f19fb035640 1 -- 172.21.15.16:0/957568478 shutdown_connections 2024-09-12T01:34:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.863+0000 7f19fb035640 1 -- 172.21.15.16:0/957568478 wait complete. 2024-09-12T01:34:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.864+0000 7f19fb035640 1 Processor -- start 2024-09-12T01:34:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.864+0000 7f19fb035640 1 -- start start 2024-09-12T01:34:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19fb035640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f406d140 0x7f19f40b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19fb035640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f19f40ceb70 0x7f19f40cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19fb035640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f415e400 0x7f19f40b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19fb035640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19f415e7e0 con 0x7f19f406d140 2024-09-12T01:34:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19fb035640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19f415eaf0 con 0x7f19f415e400 2024-09-12T01:34:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19fb035640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7f19f415f720 con 0x7f19f40ceb70 2024-09-12T01:34:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19f8dd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f406d140 0x7f19f40b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19f8dd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f406d140 0x7f19f40b66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41492/0 (socket says 172.21.15.16:41492) 2024-09-12T01:34:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19f8dd1640 1 -- 172.21.15.16:0/3264450883 learned_addr learned my addr 172.21.15.16:0/3264450883 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19f95d2640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f415e400 0x7f19f40b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19f3fff640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f19f40ceb70 0x7f19f40cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19f8dd1640 1 -- 172.21.15.16:0/3264450883 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f19f40ceb70 msgr2=0x7f19f40cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19f8dd1640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f19f40ceb70 0x7f19f40cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19f8dd1640 1 -- 172.21.15.16:0/3264450883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f415e400 msgr2=0x7f19f40b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19f8dd1640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f415e400 0x7f19f40b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19f8dd1640 1 -- 172.21.15.16:0/3264450883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f40ca550 con 0x7f19f406d140 2024-09-12T01:34:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19f95d2640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f415e400 0x7f19f40b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.865+0000 7f19f8dd1640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f406d140 0x7f19f40b66d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f19e8035040 tx=0x7f19e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.866+0000 7f19f17fa640 1 -- 172.21.15.16:0/3264450883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8037050 con 0x7f19f406d140 2024-09-12T01:34:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.866+0000 7f19f17fa640 1 -- 172.21.15.16:0/3264450883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8038040 con 0x7f19f406d140 2024-09-12T01:34:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.866+0000 7f19fb035640 1 -- 172.21.15.16:0/3264450883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f40ca7f0 con 0x7f19f406d140 2024-09-12T01:34:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.866+0000 7f19f17fa640 1 -- 172.21.15.16:0/3264450883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8004450 con 0x7f19f406d140 2024-09-12T01:34:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.866+0000 7f19fb035640 1 -- 172.21.15.16:0/3264450883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f40cb090 con 0x7f19f406d140 2024-09-12T01:34:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.867+0000 7f19fb035640 1 -- 172.21.15.16:0/3264450883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f40cabe0 con 0x7f19f406d140 2024-09-12T01:34:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.867+0000 7f19f17fa640 1 -- 172.21.15.16:0/3264450883 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19e803c6d0 con 0x7f19f406d140 2024-09-12T01:34:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.868+0000 7f19f17fa640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f19d806be30 0x7f19d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.868+0000 7f19f17fa640 1 -- 172.21.15.16:0/3264450883 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f19e80ab7f0 con 0x7f19f406d140 2024-09-12T01:34:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.868+0000 7f19f3fff640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f19d806be30 0x7f19d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.868+0000 7f19f3fff640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f19d806be30 0x7f19d806e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f19f415a8c0 tx=0x7f19e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:49.872+0000 7f19f17fa640 1 -- 172.21.15.16:0/3264450883 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f19e807a220 con 0x7f19f406d140 2024-09-12T01:34:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.048+0000 7f19fb035640 1 -- 172.21.15.16:0/3264450883 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f19f406cc00 con 0x7f19d806be30 2024-09-12T01:34:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.048+0000 7f19f17fa640 1 -- 172.21.15.16:0/3264450883 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f19f406cc00 con 0x7f19d806be30 2024-09-12T01:34:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 7 to deep-scrub 2024-09-12T01:34:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.050+0000 7f19fb035640 1 -- 172.21.15.16:0/3264450883 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f19d806be30 msgr2=0x7f19d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.050+0000 7f19fb035640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f19d806be30 0x7f19d806e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f19f415a8c0 tx=0x7f19e0008040 comp rx=0 tx=0).stop 2024-09-12T01:34:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.050+0000 7f19fb035640 1 -- 172.21.15.16:0/3264450883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f406d140 msgr2=0x7f19f40b66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.050+0000 7f19fb035640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f406d140 0x7f19f40b66d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f19e8035040 tx=0x7f19e80046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.050+0000 7f19fb035640 1 -- 172.21.15.16:0/3264450883 shutdown_connections 2024-09-12T01:34:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.050+0000 7f19fb035640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7f19d806be30 0x7f19d806e2f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.050+0000 7f19fb035640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f415e400 0x7f19f40b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.050+0000 7f19fb035640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f19f40ceb70 0x7f19f40cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.050+0000 7f19fb035640 1 --2- 172.21.15.16:0/3264450883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f406d140 0x7f19f40b66d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.050+0000 7f19fb035640 1 -- 172.21.15.16:0/3264450883 >> 172.21.15.16:0/3264450883 conn(0x7f19f406feb0 msgr2=0x7f19f40c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.050+0000 7f19fb035640 1 -- 172.21.15.16:0/3264450883 shutdown_connections 2024-09-12T01:34:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.050+0000 7f19fb035640 1 -- 172.21.15.16:0/3264450883 wait complete. 2024-09-12T01:34:50.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:34:50.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.163+0000 7feff754b640 1 Processor -- start 2024-09-12T01:34:50.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.163+0000 7feff754b640 1 -- start start 2024-09-12T01:34:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.164+0000 7feff754b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feff015e560 0x7feff00ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.164+0000 7feff754b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff00d3480 con 0x7feff00cf510 2024-09-12T01:34:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.164+0000 7feff754b640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff00d3760 con 0x7feff00d1550 2024-09-12T01:34:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.164+0000 7feff754b640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7feff00d3a40 con 0x7feff015e560 2024-09-12T01:34:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.164+0000 7feff4ae6640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7feff00cf510 0x7feff00d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48058/0 (socket says 172.21.15.16:48058) 2024-09-12T01:34:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.164+0000 7feff4ae6640 1 -- 172.21.15.16:0/1927236471 learned_addr learned my addr 172.21.15.16:0/1927236471 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-12T01:34:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.164+0000 7feff52e7640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feff015e560 0x7feff00ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1646849677 0 0) 0x7feff00d3480 con 0x7feff00cf510 2024-09-12T01:34:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefdc003520 con 0x7feff00cf510 2024-09-12T01:34:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 <== mon.1 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1078315134 0 0) 0x7feff00d3760 con 0x7feff00d1550 2024-09-12T01:34:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feff00d3480 con 0x7feff00d1550 2024-09-12T01:34:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2461517141 0 0) 0x7fefdc003520 con 0x7feff00cf510 2024-09-12T01:34:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feff015e560 msgr2=0x7feff00ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feff015e560 0x7feff00ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 >> v1:172.21.15.33:6789/0 conn(0x7feff00d1550 legacy=0x7feff00b7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff00d3d20 con 0x7feff00cf510 2024-09-12T01:34:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1709907885 0 0) 0x7fefe0001670 con 0x7feff00cf510 2024-09-12T01:34:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 >> v1:172.21.15.16:6789/0 conn(0x7feff00cf510 legacy=0x7feff00d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc003be0 0x7fefdc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefdc004800 0x7fefdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fefdc0051c0 0x7fefdc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feff00d3a40 con 0x7fefdc004800 2024-09-12T01:34:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc00c060 con 0x7fefdc003be0 2024-09-12T01:34:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fefdc00c260 con 0x7fefdc0051c0 2024-09-12T01:34:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7feff52e7640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7feff015e560 0x7feff00ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7feff4ae6640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefdc004800 0x7fefdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7feff5ae8640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fefdc0051c0 0x7fefdc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7feff52e7640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc003be0 0x7fefdc003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7feff4ae6640 1 -- 172.21.15.16:0/1927236471 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fefdc0051c0 msgr2=0x7fefdc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7feff4ae6640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fefdc0051c0 0x7fefdc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7feff4ae6640 1 -- 172.21.15.16:0/1927236471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc003be0 msgr2=0x7fefdc003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7feff4ae6640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc003be0 0x7fefdc003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7feff4ae6640 1 -- 172.21.15.16:0/1927236471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc003520 con 0x7fefdc004800 2024-09-12T01:34:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.165+0000 7feff52e7640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc003be0 0x7fefdc003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.166+0000 7feff4ae6640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefdc004800 0x7fefdc004c80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fefe000c0e0 tx=0x7fefdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b5a87485b6a81f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.166+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0007c40 con 0x7fefdc004800 2024-09-12T01:34:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.166+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe002ae50 con 0x7fefdc004800 2024-09-12T01:34:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.166+0000 7fefe7fff640 1 -- 172.21.15.16:0/1927236471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0005d50 con 0x7fefdc004800 2024-09-12T01:34:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.167+0000 7feff754b640 1 -- 172.21.15.16:0/1927236471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefdc004800 msgr2=0x7fefdc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.167+0000 7feff754b640 1 --2- 172.21.15.16:0/1927236471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefdc004800 0x7fefdc004c80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fefe000c0e0 tx=0x7fefdc005830 comp rx=0 tx=0).stop 2024-09-12T01:34:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.167+0000 7feff5ae8640 1 -- 172.21.15.16:0/1927236471 reap_dead start 2024-09-12T01:34:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.167+0000 7feff754b640 1 -- 172.21.15.16:0/1927236471 shutdown_connections 2024-09-12T01:34:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.167+0000 7feff754b640 1 -- 172.21.15.16:0/1927236471 >> 172.21.15.16:0/1927236471 conn(0x7feff0073850 msgr2=0x7feff006cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.167+0000 7feff754b640 1 -- 172.21.15.16:0/1927236471 shutdown_connections 2024-09-12T01:34:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.167+0000 7feff754b640 1 -- 172.21.15.16:0/1927236471 wait complete. 2024-09-12T01:34:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.168+0000 7feff754b640 1 Processor -- start 2024-09-12T01:34:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.168+0000 7feff754b640 1 -- start start 2024-09-12T01:34:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff754b640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fefdc004800 0x7feff014c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff754b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff00ac930 0x7feff00aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff754b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff00ab310 0x7feff00ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff754b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feff015eac0 con 0x7feff00ac930 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff754b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feff015e940 con 0x7feff00ab310 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff754b640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7feff00cf950 con 0x7fefdc004800 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff4ae6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff00ac930 0x7feff00aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff4ae6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff00ac930 0x7feff00aadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41500/0 (socket says 172.21.15.16:41500) 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff4ae6640 1 -- 172.21.15.16:0/2372460840 learned_addr learned my addr 172.21.15.16:0/2372460840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff5ae8640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff00ab310 0x7feff00ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff52e7640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fefdc004800 0x7feff014c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff4ae6640 1 -- 172.21.15.16:0/2372460840 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fefdc004800 msgr2=0x7feff014c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff4ae6640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fefdc004800 0x7feff014c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff4ae6640 1 -- 172.21.15.16:0/2372460840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff00ab310 msgr2=0x7feff00ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff4ae6640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff00ab310 0x7feff00ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff4ae6640 1 -- 172.21.15.16:0/2372460840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff00ab890 con 0x7feff00ac930 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff5ae8640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff00ab310 0x7feff00ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.169+0000 7feff4ae6640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff00ac930 0x7feff00aadd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fefe002ac20 tx=0x7fefe0001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.170+0000 7fefe5ffb640 1 -- 172.21.15.16:0/2372460840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0043070 con 0x7feff00ac930 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.170+0000 7fefe5ffb640 1 -- 172.21.15.16:0/2372460840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe0038040 con 0x7feff00ac930 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.170+0000 7fefe5ffb640 1 -- 172.21.15.16:0/2372460840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0004e30 con 0x7feff00ac930 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.170+0000 7feff754b640 1 -- 172.21.15.16:0/2372460840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff00abb70 con 0x7feff00ac930 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.170+0000 7feff754b640 1 -- 172.21.15.16:0/2372460840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff00ac130 con 0x7feff00ac930 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.171+0000 7feff754b640 1 -- 172.21.15.16:0/2372460840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff00ceee0 con 0x7feff00ac930 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.171+0000 7fefe5ffb640 1 -- 172.21.15.16:0/2372460840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fefe0007070 con 0x7feff00ac930 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.172+0000 7fefe5ffb640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fefd406be30 0x7fefd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.172+0000 7fefe5ffb640 1 -- 172.21.15.16:0/2372460840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7fefe00aa2f0 con 0x7feff00ac930 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.172+0000 7feff52e7640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fefd406be30 0x7fefd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.172+0000 7feff52e7640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fefd406be30 0x7fefd406e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fefec0093d0 tx=0x7fefec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.176+0000 7fefe5ffb640 1 -- 172.21.15.16:0/2372460840 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fefe00786c0 con 0x7feff00ac930 2024-09-12T01:34:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.348+0000 7feff754b640 1 -- 172.21.15.16:0/2372460840 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feff015d680 con 0x7fefd406be30 2024-09-12T01:34:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.348+0000 7fefe5ffb640 1 -- 172.21.15.16:0/2372460840 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13464 (secure 0 0 0) 0x7feff015d680 con 0x7fefd406be30 2024-09-12T01:34:50.348 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:34:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:34:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.350+0000 7feff754b640 1 -- 172.21.15.16:0/2372460840 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fefd406be30 msgr2=0x7fefd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.350+0000 7feff754b640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fefd406be30 0x7fefd406e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fefec0093d0 tx=0x7fefec008040 comp rx=0 tx=0).stop 2024-09-12T01:34:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.350+0000 7feff754b640 1 -- 172.21.15.16:0/2372460840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff00ac930 msgr2=0x7feff00aadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.350+0000 7feff754b640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff00ac930 0x7feff00aadd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fefe002ac20 tx=0x7fefe0001970 comp rx=0 tx=0).stop 2024-09-12T01:34:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.351+0000 7feff754b640 1 -- 172.21.15.16:0/2372460840 shutdown_connections 2024-09-12T01:34:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.351+0000 7feff754b640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fefd406be30 0x7fefd406e2f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.351+0000 7feff754b640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff00ab310 0x7feff00ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.351+0000 7feff754b640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff00ac930 0x7feff00aadd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.351+0000 7feff754b640 1 --2- 172.21.15.16:0/2372460840 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fefdc004800 0x7feff014c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.351+0000 7feff754b640 1 -- 172.21.15.16:0/2372460840 >> 172.21.15.16:0/2372460840 conn(0x7feff0073850 msgr2=0x7feff0161120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.351+0000 7feff754b640 1 -- 172.21.15.16:0/2372460840 shutdown_connections 2024-09-12T01:34:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:34:50.351+0000 7feff754b640 1 -- 172.21.15.16:0/2372460840 wait complete. 2024-09-12T01:34:50.361 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-09-12T01:34:48.491406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238052,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736668,"statfs":{"total":773094113280,"available":772850348032,"internally_reserved":0,"allocated":5283840,"data_stored":2984176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14873,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-79,"num_write_kb":-41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001609"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":212,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-12T01:34:43.551188+0000","last_change":"2024-09-12T01:33:01.096542+0000","last_active":"2024-09-12T01:34:43.551188+0000","last_peered":"2024-09-12T01:34:43.551188+0000","last_clean":"2024-09-12T01:34:43.551188+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:34:43.551188+0000","last_undegraded":"2024-09-12T01:34:43.551188+0000","last_fullsized":"2024-09-12T01:34:43.551188+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:32:58.512553+0000","last_clean_scrub_stamp":"2024-09-12T01:32:58.512553+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:48:58.741813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509465,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1105920,"data_stored":815798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1105920,"data_stored":815798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:34:50.364 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-12T01:32:58.512553+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=1, tm_min=32, tm_sec=58, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=1, tm_min=34, tm_sec=45, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T01:34:50.365 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-12T01:35:10.366 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:35:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.482+0000 7fb8cbf78640 1 Processor -- start 2024-09-12T01:35:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.482+0000 7fb8cbf78640 1 -- start start 2024-09-12T01:35:10.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.483+0000 7fb8cbf78640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8c40d0350 0x7fb8c406cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:10.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.483+0000 7fb8cbf78640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb8c40b1930 0x7fb8c40b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:10.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.483+0000 7fb8cbf78640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8c415e7f0 con 0x7fb8c409aa30 2024-09-12T01:35:10.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.483+0000 7fb8cbf78640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c40d1a50 con 0x7fb8c40d0350 2024-09-12T01:35:10.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.483+0000 7fb8cbf78640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c40d09b0 con 0x7fb8c40b1930 2024-09-12T01:35:10.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.483+0000 7fb8c9513640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8c40d0350 0x7fb8c406cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:10.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.483+0000 7fb8c9513640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8c40d0350 0x7fb8c406cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41186/0 (socket says 172.21.15.16:41186) 2024-09-12T01:35:10.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.483+0000 7fb8c9513640 1 -- 172.21.15.16:0/3635140776 learned_addr learned my addr 172.21.15.16:0/3635140776 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:35:10.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.483+0000 7fb8ca515640 1 --2- 172.21.15.16:0/3635140776 >> v2:172.21.15.33:3300/0 conn(0x7fb8c40b1930 0x7fb8c40b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:10.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.484+0000 7fb8c9513640 1 -- 172.21.15.16:0/3635140776 >> v2:172.21.15.33:3300/0 conn(0x7fb8c40b1930 msgr2=0x7fb8c40b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:10.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.484+0000 7fb8c9513640 1 --2- 172.21.15.16:0/3635140776 >> v2:172.21.15.33:3300/0 conn(0x7fb8c40b1930 0x7fb8c40b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.484+0000 7fb8c9513640 1 -- 172.21.15.16:0/3635140776 >> v1:172.21.15.33:6789/0 conn(0x7fb8c409aa30 legacy=0x7fb8c409ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:10.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.484+0000 7fb8c9513640 1 -- 172.21.15.16:0/3635140776 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c40d0c90 con 0x7fb8c40d0350 2024-09-12T01:35:10.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.484+0000 7fb8c9513640 1 --2- 172.21.15.16:0/3635140776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c40d0350 0x7fb8c406cc00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4162f90 tx=0x7fb8b402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90328ea90da0174e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:10.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.484+0000 7fb8c8d12640 1 -- 172.21.15.16:0/3635140776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4037070 con 0x7fb8c40d0350 2024-09-12T01:35:10.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.485+0000 7fb8c8d12640 1 -- 172.21.15.16:0/3635140776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b4032440 con 0x7fb8c40d0350 2024-09-12T01:35:10.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.485+0000 7fb8c8d12640 1 -- 172.21.15.16:0/3635140776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4032810 con 0x7fb8c40d0350 2024-09-12T01:35:10.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.485+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3635140776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c40d0350 msgr2=0x7fb8c406cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:10.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.485+0000 7fb8cbf78640 1 --2- 172.21.15.16:0/3635140776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c40d0350 0x7fb8c406cc00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4162f90 tx=0x7fb8b402a230 comp rx=0 tx=0).stop 2024-09-12T01:35:10.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.485+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3635140776 shutdown_connections 2024-09-12T01:35:10.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.485+0000 7fb8cbf78640 1 --2- 172.21.15.16:0/3635140776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c40d0350 0x7fb8c406cc00 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.485+0000 7fb8cbf78640 1 --2- 172.21.15.16:0/3635140776 >> v2:172.21.15.33:3300/0 conn(0x7fb8c40b1930 0x7fb8c40b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.485+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3635140776 >> 172.21.15.16:0/3635140776 conn(0x7fb8c40b3cf0 msgr2=0x7fb8c40b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:10.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.485+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3635140776 shutdown_connections 2024-09-12T01:35:10.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.485+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3635140776 wait complete. 2024-09-12T01:35:10.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.486+0000 7fb8cbf78640 1 Processor -- start 2024-09-12T01:35:10.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.486+0000 7fb8cbf78640 1 -- start start 2024-09-12T01:35:10.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.487+0000 7fb8cbf78640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c409aa30 0x7fb8c415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:10.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.487+0000 7fb8cbf78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c40b1930 0x7fb8c41f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.487+0000 7fb8cbf78640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb8c40d0350 0x7fb8c415e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.487+0000 7fb8cbf78640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c40d1530 con 0x7fb8c40b1930 2024-09-12T01:35:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.487+0000 7fb8cbf78640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c40d1250 con 0x7fb8c409aa30 2024-09-12T01:35:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.487+0000 7fb8cbf78640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c406d140 con 0x7fb8c40d0350 2024-09-12T01:35:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.487+0000 7fb8c9513640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c40b1930 0x7fb8c41f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.487+0000 7fb8c9513640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c40b1930 0x7fb8c41f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41194/0 (socket says 172.21.15.16:41194) 2024-09-12T01:35:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.487+0000 7fb8c9513640 1 -- 172.21.15.16:0/3640647507 learned_addr learned my addr 172.21.15.16:0/3640647507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:35:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.487+0000 7fb8c9d14640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c409aa30 0x7fb8c415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.487+0000 7fb8ca515640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb8c40d0350 0x7fb8c415e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.488+0000 7fb8c9513640 1 -- 172.21.15.16:0/3640647507 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb8c40d0350 msgr2=0x7fb8c415e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.488+0000 7fb8c9513640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb8c40d0350 0x7fb8c415e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.488+0000 7fb8c9513640 1 -- 172.21.15.16:0/3640647507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c409aa30 msgr2=0x7fb8c415a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.488+0000 7fb8c9513640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c409aa30 0x7fb8c415a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.488+0000 7fb8c9513640 1 -- 172.21.15.16:0/3640647507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c415a930 con 0x7fb8c40b1930 2024-09-12T01:35:10.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.488+0000 7fb8c9513640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c40b1930 0x7fb8c41f4820 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb8b402a270 tx=0x7fb8b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:10.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.488+0000 7fb8ba7fc640 1 -- 172.21.15.16:0/3640647507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4041020 con 0x7fb8c40b1930 2024-09-12T01:35:10.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.488+0000 7fb8ba7fc640 1 -- 172.21.15.16:0/3640647507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b40306c0 con 0x7fb8c40b1930 2024-09-12T01:35:10.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.488+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3640647507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c41f4d60 con 0x7fb8c40b1930 2024-09-12T01:35:10.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.488+0000 7fb8ba7fc640 1 -- 172.21.15.16:0/3640647507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4037040 con 0x7fb8c40b1930 2024-09-12T01:35:10.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.488+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3640647507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c4160f20 con 0x7fb8c40b1930 2024-09-12T01:35:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.489+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3640647507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c406cc00 con 0x7fb8c40b1930 2024-09-12T01:35:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.490+0000 7fb8ba7fc640 1 -- 172.21.15.16:0/3640647507 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb8b4044020 con 0x7fb8c40b1930 2024-09-12T01:35:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.490+0000 7fb8ba7fc640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb8a806be30 0x7fb8a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.490+0000 7fb8ba7fc640 1 -- 172.21.15.16:0/3640647507 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7fb8b40ab4f0 con 0x7fb8c40b1930 2024-09-12T01:35:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.490+0000 7fb8c9d14640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb8a806be30 0x7fb8a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.491+0000 7fb8c9d14640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb8a806be30 0x7fb8a806e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb8c415d790 tx=0x7fb8c0007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.495+0000 7fb8ba7fc640 1 -- 172.21.15.16:0/3640647507 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb8b404b030 con 0x7fb8c40b1930 2024-09-12T01:35:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.669+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3640647507 --> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8c40b8180 con 0x7fb8a806be30 2024-09-12T01:35:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.670+0000 7fb8ba7fc640 1 -- 172.21.15.16:0/3640647507 <== mgr.4107 v2:172.21.15.16:6832/2688455607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13359 (secure 0 0 0) 0x7fb8c40b8180 con 0x7fb8a806be30 2024-09-12T01:35:10.670 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:35:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T01:35:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.672+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3640647507 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb8a806be30 msgr2=0x7fb8a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.672+0000 7fb8cbf78640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb8a806be30 0x7fb8a806e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb8c415d790 tx=0x7fb8c0007e30 comp rx=0 tx=0).stop 2024-09-12T01:35:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.672+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3640647507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c40b1930 msgr2=0x7fb8c41f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.672+0000 7fb8cbf78640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c40b1930 0x7fb8c41f4820 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb8b402a270 tx=0x7fb8b40046c0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.672+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3640647507 shutdown_connections 2024-09-12T01:35:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.672+0000 7fb8cbf78640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb8a806be30 0x7fb8a806e2f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.672+0000 7fb8cbf78640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb8c40d0350 0x7fb8c415e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.672+0000 7fb8cbf78640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c40b1930 0x7fb8c41f4820 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.672+0000 7fb8cbf78640 1 --2- 172.21.15.16:0/3640647507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c409aa30 0x7fb8c415a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.672+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3640647507 >> 172.21.15.16:0/3640647507 conn(0x7fb8c40b3cf0 msgr2=0x7fb8c40b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.672+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3640647507 shutdown_connections 2024-09-12T01:35:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.672+0000 7fb8cbf78640 1 -- 172.21.15.16:0/3640647507 wait complete. 2024-09-12T01:35:10.682 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-12T01:35:10.496101+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238040,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736680,"statfs":{"total":773094113280,"available":772850360320,"internally_reserved":0,"allocated":5283840,"data_stored":2984176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002869"},"pg_stats":[{"pgid":"1.0","version":"44'119","reported_seq":224,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-12T01:34:50.649302+0000","last_change":"2024-09-12T01:34:50.649302+0000","last_active":"2024-09-12T01:34:50.649302+0000","last_peered":"2024-09-12T01:34:50.649302+0000","last_clean":"2024-09-12T01:34:50.649302+0000","last_became_active":"2024-09-12T01:33:01.095556+0000","last_became_peered":"2024-09-12T01:33:01.095556+0000","last_unstale":"2024-09-12T01:34:50.649302+0000","last_undegraded":"2024-09-12T01:34:50.649302+0000","last_fullsized":"2024-09-12T01:34:50.649302+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'119","last_scrub_stamp":"2024-09-12T01:34:50.649175+0000","last_deep_scrub":"44'119","last_deep_scrub_stamp":"2024-09-12T01:34:50.649175+0000","last_clean_scrub_stamp":"2024-09-12T01:34:50.649175+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:54:30.308588+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":815798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":512000,"data_stored":225430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1105920,"data_stored":815798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:35:10.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-12T01:35:10.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.786+0000 7fb78965e640 1 Processor -- start 2024-09-12T01:35:10.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.786+0000 7fb78965e640 1 -- start start 2024-09-12T01:35:10.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.787+0000 7fb78965e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb78415e480 0x7fb7840d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:10.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.787+0000 7fb78965e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7840b0370 0x7fb78406baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:10.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.787+0000 7fb78965e640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7840cdc80 con 0x7fb78406c030 2024-09-12T01:35:10.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.787+0000 7fb78965e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb7840b3dc0 con 0x7fb7840b0370 2024-09-12T01:35:10.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.787+0000 7fb78965e640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb78406c480 con 0x7fb78415e480 2024-09-12T01:35:10.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.787+0000 7fb77bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7840b0370 0x7fb78406baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.787+0000 7fb77bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7840b0370 0x7fb78406baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41196/0 (socket says 172.21.15.16:41196) 2024-09-12T01:35:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.787+0000 7fb77bfff640 1 -- 172.21.15.16:0/1310353324 learned_addr learned my addr 172.21.15.16:0/1310353324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:35:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.787+0000 7fb77bfff640 1 -- 172.21.15.16:0/1310353324 >> v2:172.21.15.33:3300/0 conn(0x7fb78415e480 msgr2=0x7fb7840d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.787+0000 7fb77bfff640 1 --2- 172.21.15.16:0/1310353324 >> v2:172.21.15.33:3300/0 conn(0x7fb78415e480 0x7fb7840d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.787+0000 7fb77bfff640 1 -- 172.21.15.16:0/1310353324 >> v1:172.21.15.33:6789/0 conn(0x7fb78406c030 legacy=0x7fb784162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.787+0000 7fb77bfff640 1 -- 172.21.15.16:0/1310353324 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7840cdf40 con 0x7fb7840b0370 2024-09-12T01:35:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.788+0000 7fb77bfff640 1 --2- 172.21.15.16:0/1310353324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7840b0370 0x7fb78406baf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb784073de0 tx=0x7fb76c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4ffcb0dd71731c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.788+0000 7fb7827fc640 1 -- 172.21.15.16:0/1310353324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c037070 con 0x7fb7840b0370 2024-09-12T01:35:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.788+0000 7fb7827fc640 1 -- 172.21.15.16:0/1310353324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76c02ae20 con 0x7fb7840b0370 2024-09-12T01:35:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.788+0000 7fb7827fc640 1 -- 172.21.15.16:0/1310353324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c032660 con 0x7fb7840b0370 2024-09-12T01:35:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.789+0000 7fb78965e640 1 -- 172.21.15.16:0/1310353324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7840b0370 msgr2=0x7fb78406baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.789+0000 7fb78965e640 1 --2- 172.21.15.16:0/1310353324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7840b0370 0x7fb78406baf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb784073de0 tx=0x7fb76c02a110 comp rx=0 tx=0).stop 2024-09-12T01:35:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.789+0000 7fb78965e640 1 -- 172.21.15.16:0/1310353324 shutdown_connections 2024-09-12T01:35:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.789+0000 7fb78965e640 1 --2- 172.21.15.16:0/1310353324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7840b0370 0x7fb78406baf0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.789+0000 7fb78965e640 1 --2- 172.21.15.16:0/1310353324 >> v2:172.21.15.33:3300/0 conn(0x7fb78415e480 0x7fb7840d0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.789+0000 7fb78965e640 1 -- 172.21.15.16:0/1310353324 >> 172.21.15.16:0/1310353324 conn(0x7fb78409aa30 msgr2=0x7fb7840b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.789+0000 7fb78965e640 1 -- 172.21.15.16:0/1310353324 shutdown_connections 2024-09-12T01:35:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.789+0000 7fb78965e640 1 -- 172.21.15.16:0/1310353324 wait complete. 2024-09-12T01:35:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.790+0000 7fb78965e640 1 Processor -- start 2024-09-12T01:35:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.790+0000 7fb78965e640 1 -- start start 2024-09-12T01:35:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb78965e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78406c030 0x7fb7841f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb78965e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7840b0370 0x7fb7840c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb78965e640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb78415e480 0x7fb78415a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb78965e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb78415e860 con 0x7fb7840b0370 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb78965e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb7840ce500 con 0x7fb78406c030 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb78965e640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fb7840af150 con 0x7fb78415e480 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb77bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7840b0370 0x7fb7840c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb77bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7840b0370 0x7fb7840c7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41204/0 (socket says 172.21.15.16:41204) 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb77bfff640 1 -- 172.21.15.16:0/1605208765 learned_addr learned my addr 172.21.15.16:0/1605208765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb7837fe640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb78415e480 0x7fb78415a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb782ffd640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78406c030 0x7fb7841f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb77bfff640 1 -- 172.21.15.16:0/1605208765 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb78415e480 msgr2=0x7fb78415a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb77bfff640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb78415e480 0x7fb78415a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb77bfff640 1 -- 172.21.15.16:0/1605208765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78406c030 msgr2=0x7fb7841f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb77bfff640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78406c030 0x7fb7841f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb77bfff640 1 -- 172.21.15.16:0/1605208765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7840c5ca0 con 0x7fb7840b0370 2024-09-12T01:35:10.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.791+0000 7fb782ffd640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78406c030 0x7fb7841f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:35:10.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.792+0000 7fb77bfff640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7840b0370 0x7fb7840c7a00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb76c02a150 tx=0x7fb76c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:10.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.792+0000 7fb77b7fe640 1 -- 172.21.15.16:0/1605208765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c042070 con 0x7fb7840b0370 2024-09-12T01:35:10.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.792+0000 7fb77b7fe640 1 -- 172.21.15.16:0/1605208765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76c03e670 con 0x7fb7840b0370 2024-09-12T01:35:10.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.792+0000 7fb78965e640 1 -- 172.21.15.16:0/1605208765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7840c5f80 con 0x7fb7840b0370 2024-09-12T01:35:10.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.792+0000 7fb77b7fe640 1 -- 172.21.15.16:0/1605208765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c037040 con 0x7fb7840b0370 2024-09-12T01:35:10.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.792+0000 7fb78965e640 1 -- 172.21.15.16:0/1605208765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7840c6540 con 0x7fb7840b0370 2024-09-12T01:35:10.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.793+0000 7fb78965e640 1 -- 172.21.15.16:0/1605208765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb78406baf0 con 0x7fb7840b0370 2024-09-12T01:35:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.793+0000 7fb77b7fe640 1 -- 172.21.15.16:0/1605208765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb76c045070 con 0x7fb7840b0370 2024-09-12T01:35:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.794+0000 7fb77b7fe640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb76406be30 0x7fb76406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.794+0000 7fb77b7fe640 1 -- 172.21.15.16:0/1605208765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7fb76c0ab350 con 0x7fb7840b0370 2024-09-12T01:35:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.794+0000 7fb782ffd640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb76406be30 0x7fb76406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.794+0000 7fb782ffd640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb76406be30 0x7fb76406e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb774009890 tx=0x7fb774008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:10.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.798+0000 7fb77b7fe640 1 -- 172.21.15.16:0/1605208765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb76c02ae20 con 0x7fb7840b0370 2024-09-12T01:35:10.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.976+0000 7fb78965e640 1 -- 172.21.15.16:0/1605208765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fb7840c6780 con 0x7fb7840b0370 2024-09-12T01:35:10.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.979+0000 7fb77b7fe640 1 -- 172.21.15.16:0/1605208765 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7fb76c0797d0 con 0x7fb7840b0370 2024-09-12T01:35:10.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.980+0000 7fb77b7fe640 1 -- 172.21.15.16:0/1605208765 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb76c07dad0 con 0x7fb7840b0370 2024-09-12T01:35:10.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.982+0000 7fb78965e640 1 -- 172.21.15.16:0/1605208765 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb76406be30 msgr2=0x7fb76406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:10.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.982+0000 7fb78965e640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb76406be30 0x7fb76406e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb774009890 tx=0x7fb774008040 comp rx=0 tx=0).stop 2024-09-12T01:35:10.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.982+0000 7fb78965e640 1 -- 172.21.15.16:0/1605208765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7840b0370 msgr2=0x7fb7840c7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:10.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.982+0000 7fb78965e640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7840b0370 0x7fb7840c7a00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb76c02a150 tx=0x7fb76c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.982+0000 7fb78965e640 1 -- 172.21.15.16:0/1605208765 shutdown_connections 2024-09-12T01:35:10.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.982+0000 7fb78965e640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fb76406be30 0x7fb76406e2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.982+0000 7fb78965e640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fb78415e480 0x7fb78415a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.982+0000 7fb78965e640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7840b0370 0x7fb7840c7a00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.982+0000 7fb78965e640 1 --2- 172.21.15.16:0/1605208765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78406c030 0x7fb7841f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:10.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.982+0000 7fb78965e640 1 -- 172.21.15.16:0/1605208765 >> 172.21.15.16:0/1605208765 conn(0x7fb78409aa30 msgr2=0x7fb7840d16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:10.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.982+0000 7fb78965e640 1 -- 172.21.15.16:0/1605208765 shutdown_connections 2024-09-12T01:35:10.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:10.982+0000 7fb78965e640 1 -- 172.21.15.16:0/1605208765 wait complete. 2024-09-12T01:35:10.993 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T01:35:10.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T01:35:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.095+0000 7fe175d66640 1 Processor -- start 2024-09-12T01:35:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.095+0000 7fe175d66640 1 -- start start 2024-09-12T01:35:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.096+0000 7fe175d66640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe17009aa30 0x7fe1700d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.096+0000 7fe175d66640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe17009ae10 0x7fe17006cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.096+0000 7fe175d66640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe17015e7f0 con 0x7fe170161ef0 2024-09-12T01:35:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.096+0000 7fe175d66640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe1700d1a50 con 0x7fe17009aa30 2024-09-12T01:35:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.096+0000 7fe175d66640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fe1700d09b0 con 0x7fe17009ae10 2024-09-12T01:35:11.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.096+0000 7fe16f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe17009aa30 0x7fe1700d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:11.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.096+0000 7fe16f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe17009aa30 0x7fe1700d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41220/0 (socket says 172.21.15.16:41220) 2024-09-12T01:35:11.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.096+0000 7fe16f7fe640 1 -- 172.21.15.16:0/55536070 learned_addr learned my addr 172.21.15.16:0/55536070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:35:11.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.096+0000 7fe16f7fe640 1 -- 172.21.15.16:0/55536070 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe17009ae10 msgr2=0x7fe17006cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:11.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.096+0000 7fe16f7fe640 1 --2- 172.21.15.16:0/55536070 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe17009ae10 0x7fe17006cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:11.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.096+0000 7fe16f7fe640 1 -- 172.21.15.16:0/55536070 >> v1:172.21.15.33:6789/0 conn(0x7fe170161ef0 legacy=0x7fe170162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:11.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.096+0000 7fe16f7fe640 1 -- 172.21.15.16:0/55536070 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1700d0c90 con 0x7fe17009aa30 2024-09-12T01:35:11.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.097+0000 7fe16f7fe640 1 --2- 172.21.15.16:0/55536070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17009aa30 0x7fe1700d0350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe160009870 tx=0x7fe16002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c51559dae590f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:11.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.097+0000 7fe16e7fc640 1 -- 172.21.15.16:0/55536070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160037070 con 0x7fe17009aa30 2024-09-12T01:35:11.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.097+0000 7fe16e7fc640 1 -- 172.21.15.16:0/55536070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe160032440 con 0x7fe17009aa30 2024-09-12T01:35:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.097+0000 7fe16e7fc640 1 -- 172.21.15.16:0/55536070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160032830 con 0x7fe17009aa30 2024-09-12T01:35:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.098+0000 7fe175d66640 1 -- 172.21.15.16:0/55536070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17009aa30 msgr2=0x7fe1700d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.098+0000 7fe175d66640 1 --2- 172.21.15.16:0/55536070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17009aa30 0x7fe1700d0350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe160009870 tx=0x7fe16002a210 comp rx=0 tx=0).stop 2024-09-12T01:35:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.098+0000 7fe175d66640 1 -- 172.21.15.16:0/55536070 shutdown_connections 2024-09-12T01:35:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.098+0000 7fe175d66640 1 --2- 172.21.15.16:0/55536070 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe17009ae10 0x7fe17006cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.098+0000 7fe175d66640 1 --2- 172.21.15.16:0/55536070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17009aa30 0x7fe1700d0350 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.098+0000 7fe175d66640 1 -- 172.21.15.16:0/55536070 >> 172.21.15.16:0/55536070 conn(0x7fe1700b3cf0 msgr2=0x7fe1700b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.098+0000 7fe175d66640 1 -- 172.21.15.16:0/55536070 shutdown_connections 2024-09-12T01:35:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.098+0000 7fe175d66640 1 -- 172.21.15.16:0/55536070 wait complete. 2024-09-12T01:35:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.099+0000 7fe175d66640 1 Processor -- start 2024-09-12T01:35:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.099+0000 7fe175d66640 1 -- start start 2024-09-12T01:35:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe175d66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17009aa30 0x7fe17015c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe175d66640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe17009ae10 0x7fe1700cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe175d66640 1 --2- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe170161ef0 0x7fe1700b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe175d66640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1700d1530 con 0x7fe17009aa30 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe175d66640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1700d1250 con 0x7fe17009ae10 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe175d66640 1 -- --> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] -- mon_getmap magic: 0 -- 0x7fe1700b2940 con 0x7fe170161ef0 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe16f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17009aa30 0x7fe17015c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe16f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17009aa30 0x7fe17015c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41234/0 (socket says 172.21.15.16:41234) 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe16f7fe640 1 -- 172.21.15.16:0/10998567 learned_addr learned my addr 172.21.15.16:0/10998567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe16f7fe640 1 -- 172.21.15.16:0/10998567 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe170161ef0 msgr2=0x7fe1700b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe16f7fe640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe170161ef0 0x7fe1700b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe16f7fe640 1 -- 172.21.15.16:0/10998567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe17009ae10 msgr2=0x7fe1700cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe16effd640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe17009ae10 0x7fe1700cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe16f7fe640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe17009ae10 0x7fe1700cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe16f7fe640 1 -- 172.21.15.16:0/10998567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1700b5410 con 0x7fe17009aa30 2024-09-12T01:35:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.100+0000 7fe16effd640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe17009ae10 0x7fe1700cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:35:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.101+0000 7fe16f7fe640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17009aa30 0x7fe17015c2f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe160000c00 tx=0x7fe1600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.101+0000 7fe157fff640 1 -- 172.21.15.16:0/10998567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160037040 con 0x7fe17009aa30 2024-09-12T01:35:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.101+0000 7fe157fff640 1 -- 172.21.15.16:0/10998567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe160042070 con 0x7fe17009aa30 2024-09-12T01:35:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.101+0000 7fe157fff640 1 -- 172.21.15.16:0/10998567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160030bf0 con 0x7fe17009aa30 2024-09-12T01:35:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.101+0000 7fe175d66640 1 -- 172.21.15.16:0/10998567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1700b56f0 con 0x7fe17009aa30 2024-09-12T01:35:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.101+0000 7fe175d66640 1 -- 172.21.15.16:0/10998567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1700b5cb0 con 0x7fe17009aa30 2024-09-12T01:35:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.102+0000 7fe175d66640 1 -- 172.21.15.16:0/10998567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe17009b280 con 0x7fe17009aa30 2024-09-12T01:35:11.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.102+0000 7fe157fff640 1 -- 172.21.15.16:0/10998567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe160047460 con 0x7fe17009aa30 2024-09-12T01:35:11.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.103+0000 7fe157fff640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe15c06be30 0x7fe15c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:11.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.103+0000 7fe157fff640 1 -- 172.21.15.16:0/10998567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7fe1600ab970 con 0x7fe17009aa30 2024-09-12T01:35:11.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.103+0000 7fe16effd640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe15c06be30 0x7fe15c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:11.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.103+0000 7fe16effd640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe15c06be30 0x7fe15c06e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe164006fd0 tx=0x7fe164008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.107+0000 7fe157fff640 1 -- 172.21.15.16:0/10998567 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe160045360 con 0x7fe17009aa30 2024-09-12T01:35:11.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.318+0000 7fe175d66640 1 -- 172.21.15.16:0/10998567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1700b8180 con 0x7fe17009aa30 2024-09-12T01:35:11.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.318+0000 7fe157fff640 1 -- 172.21.15.16:0/10998567 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+3902 (secure 0 0 0) 0x7fe16007a440 con 0x7fe17009aa30 2024-09-12T01:35:11.318 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:35:11.318 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-12T01:34:40:535214+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/842803487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":842803487},{"type":"v1","addr":"172.21.15.33:6833","nonce":842803487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/70052721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":70052721},{"type":"v1","addr":"172.21.15.33:6835","nonce":70052721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3777111462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3777111462},{"type":"v1","addr":"172.21.15.16:6837","nonce":3777111462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5259,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2851755875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2851755875},{"type":"v1","addr":"172.21.15.16:6835","nonce":2851755875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[]} 2024-09-12T01:35:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-12T01:35:11.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.320+0000 7fe175d66640 1 -- 172.21.15.16:0/10998567 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe15c06be30 msgr2=0x7fe15c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:11.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.320+0000 7fe175d66640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe15c06be30 0x7fe15c06e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe164006fd0 tx=0x7fe164008040 comp rx=0 tx=0).stop 2024-09-12T01:35:11.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.320+0000 7fe175d66640 1 -- 172.21.15.16:0/10998567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17009aa30 msgr2=0x7fe17015c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:11.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.320+0000 7fe175d66640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17009aa30 0x7fe17015c2f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe160000c00 tx=0x7fe1600046c0 comp rx=0 tx=0).stop 2024-09-12T01:35:11.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.320+0000 7fe175d66640 1 -- 172.21.15.16:0/10998567 shutdown_connections 2024-09-12T01:35:11.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.320+0000 7fe175d66640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.16:6832/2688455607,v1:172.21.15.16:6833/2688455607] conn(0x7fe15c06be30 0x7fe15c06e2f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:11.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.320+0000 7fe175d66640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7fe170161ef0 0x7fe1700b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:11.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.320+0000 7fe175d66640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe17009ae10 0x7fe1700cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:11.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.320+0000 7fe175d66640 1 --2- 172.21.15.16:0/10998567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17009aa30 0x7fe17015c2f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:11.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.320+0000 7fe175d66640 1 -- 172.21.15.16:0/10998567 >> 172.21.15.16:0/10998567 conn(0x7fe1700b3cf0 msgr2=0x7fe170162340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:11.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.320+0000 7fe175d66640 1 -- 172.21.15.16:0/10998567 shutdown_connections 2024-09-12T01:35:11.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:35:11.320+0000 7fe175d66640 1 -- 172.21.15.16:0/10998567 wait complete. 2024-09-12T01:35:11.331 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-12T01:35:11.332 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-12T01:35:11.332 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T01:35:11.332 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:11.341 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T01:35:11.341 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T01:35:11.341 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:11.391 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T01:35:11.391 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T01:35:11.392 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:11.404 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T01:35:11.405 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T01:35:11.405 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:11.457 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T01:35:11.458 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-12T01:35:11.458 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-12T01:35:11.458 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:11.486 INFO:tasks.ceph.osd.0:Stopped 2024-09-12T01:35:11.486 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-12T01:35:11.487 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:11.556 INFO:tasks.ceph.osd.1:Stopped 2024-09-12T01:35:11.557 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-12T01:35:11.557 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:11.624 INFO:tasks.ceph.osd.2:Stopped 2024-09-12T01:35:11.625 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-12T01:35:11.625 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:11.697 INFO:tasks.ceph.osd.3:Stopped 2024-09-12T01:35:11.698 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-12T01:35:11.698 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:11.728 INFO:tasks.ceph.osd.4:Stopped 2024-09-12T01:35:11.728 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-12T01:35:11.728 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:11.807 INFO:tasks.ceph.osd.5:Stopped 2024-09-12T01:35:11.807 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-12T01:35:11.807 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:11.880 INFO:tasks.ceph.osd.6:Stopped 2024-09-12T01:35:11.880 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-12T01:35:11.880 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:11.953 INFO:tasks.ceph.osd.7:Stopped 2024-09-12T01:35:11.953 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-12T01:35:11.953 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-12T01:35:11.953 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:11.981 INFO:tasks.ceph.mgr.y:Stopped 2024-09-12T01:35:11.981 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-12T01:35:11.981 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:12.021 INFO:tasks.ceph.mgr.x:Stopped 2024-09-12T01:35:12.021 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-12T01:35:12.021 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-12T01:35:12.021 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:12.033 INFO:tasks.ceph.mon.a:Stopped 2024-09-12T01:35:12.033 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-12T01:35:12.033 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:12.074 INFO:tasks.ceph.mon.b:Stopped 2024-09-12T01:35:12.074 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-12T01:35:12.074 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:35:12.128 INFO:tasks.ceph.mon.c:Stopped 2024-09-12T01:35:12.129 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-12T01:35:12.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'Replacing daemon mds.a' | head -n 1 2024-09-12T01:35:12.165 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:35:12.165 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-12T01:35:12.364 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:35:12.364 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-12T01:35:12.458 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:35:12.458 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-12T01:35:12.516 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:35:12.517 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-12T01:35:12.572 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:35:12.572 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-12T01:35:12.690 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:35:12.690 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-12T01:35:12.777 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:35:12.777 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-12T01:35:12.835 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:35:12.835 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-12T01:35:12.892 INFO:tasks.ceph:Archiving mon data... 2024-09-12T01:35:12.894 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901553/data/mon.a.tgz 2024-09-12T01:35:12.894 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-12T01:35:12.910 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.kvDMl2kERQ 2024-09-12T01:35:12.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.kvDMl2kERQ 2024-09-12T01:35:12.973 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:35:12.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:35:13.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.kvDMl2kERQ 2024-09-12T01:35:13.181 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.kvDMl2kERQ is 306KB 2024-09-12T01:35:13.278 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.kvDMl2kERQ 2024-09-12T01:35:13.294 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901553/data/mon.b.tgz 2024-09-12T01:35:13.294 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-09-12T01:35:13.313 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.3rTqLtPgOJ 2024-09-12T01:35:13.313 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.3rTqLtPgOJ 2024-09-12T01:35:13.486 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.3rTqLtPgOJ 2024-09-12T01:35:13.571 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.3rTqLtPgOJ is 411KB 2024-09-12T01:35:13.669 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.3rTqLtPgOJ 2024-09-12T01:35:13.684 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901553/data/mon.c.tgz 2024-09-12T01:35:13.685 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-09-12T01:35:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.Hr9NfIpUhy 2024-09-12T01:35:13.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Hr9NfIpUhy 2024-09-12T01:35:13.882 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.Hr9NfIpUhy 2024-09-12T01:35:13.967 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.Hr9NfIpUhy is 411KB 2024-09-12T01:35:14.064 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.Hr9NfIpUhy 2024-09-12T01:35:14.079 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-12T01:35:14.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T01:35:14.082 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T01:35:14.168 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-12T01:35:14.199 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-12T01:35:14.199 INFO:tasks.ceph:Archiving crash dumps... 2024-09-12T01:35:14.200 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901553/remote/smithi016/crash 2024-09-12T01:35:14.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-12T01:35:14.238 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901553/remote/smithi033/crash 2024-09-12T01:35:14.238 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-12T01:35:14.264 INFO:tasks.ceph:Shutting down logrotate 2024-09-12T01:35:14.264 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:35:14.282 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T01:35:14.356 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-12T01:35:14.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-12T01:35:14.411 INFO:tasks.ceph:Compressing logs... 2024-09-12T01:35:14.411 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T01:35:14.428 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T01:35:14.453 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-12T01:35:14.454 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42911.log 2024-09-12T01:35:14.454 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49516.log 2024-09-12T01:35:14.454 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.42911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46217.log 2024-09-12T01:35:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49516.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42911.log.gz 2024-09-12T01:35:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41496.log 2024-09-12T01:35:14.479 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49516.log.gz 2024-09-12T01:35:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43583.log 2024-09-12T01:35:14.479 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46217.log.gz 2024-09-12T01:35:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49689.log 2024-09-12T01:35:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47572.log 2024-09-12T01:35:14.479 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41496.log.gz 2024-09-12T01:35:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49689.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43583.log.gz 2024-09-12T01:35:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50319.log 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49689.log.gz 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48762.log 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47572.log.gz 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50109.log 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48762.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50319.log.gz 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47758.log 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48762.log.gz 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46553.log 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50109.log.gz 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45387.log 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46553.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47758.log.gz 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49474.log 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46553.log.gz 2024-09-12T01:35:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48446.log 2024-09-12T01:35:14.481 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45387.log.gz 2024-09-12T01:35:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49899.log 2024-09-12T01:35:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48446.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49474.log.gz 2024-09-12T01:35:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50361.log 2024-09-12T01:35:14.481 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48446.log.gz 2024-09-12T01:35:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40706.log 2024-09-12T01:35:14.481 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49899.log.gz 2024-09-12T01:35:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46469.log 2024-09-12T01:35:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40706.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50361.log.gz 2024-09-12T01:35:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45112.log 2024-09-12T01:35:14.481 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40706.log.gz 2024-09-12T01:35:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41370.log 2024-09-12T01:35:14.481 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46469.log.gz 2024-09-12T01:35:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.2.45112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43233.log 2024-09-12T01:35:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46511.log 2024-09-12T01:35:14.482 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41370.log.gz 2024-09-12T01:35:14.482 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.2.45112.log.gz 2024-09-12T01:35:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41244.log 2024-09-12T01:35:14.482 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43233.log.gz 2024-09-12T01:35:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42732.log 2024-09-12T01:35:14.482 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46511.log.gz 2024-09-12T01:35:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46385.log 2024-09-12T01:35:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42732.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41244.log.gz 2024-09-12T01:35:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42914.log 2024-09-12T01:35:14.482 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42732.log.gz 2024-09-12T01:35:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44034.log 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46385.log.gz 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42910.log 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43908.log 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42914.log.gz 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0%/var/log/ceph/ceph-client.admin.42910.log: -- replaced with /var/log/ceph/ceph-client.admin.44034.log.gz 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42910.log.gz 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46343.log 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47842.log 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43908.log.gz 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43244.log 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46343.log.gz 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41056.log 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47842.log.gz 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42480.log 2024-09-12T01:35:14.483 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43244.log.gz 2024-09-12T01:35:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45513.log 2024-09-12T01:35:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41056.log: /var/log/ceph/ceph-client.admin.42480.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30154.log 2024-09-12T01:35:14.484 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41056.log.gz 2024-09-12T01:35:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45513.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42480.log.gz 2024-09-12T01:35:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45640.log 2024-09-12T01:35:14.484 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45513.log.gz 2024-09-12T01:35:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.tmp-client.admin.30154.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30154.log.gz 2024-09-12T01:35:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49287.log 2024-09-12T01:35:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49245.log 2024-09-12T01:35:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46056.log 2024-09-12T01:35:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49287.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45640.log.gz 2024-09-12T01:35:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49245.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49287.log.gz 2024-09-12T01:35:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45924.log 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49245.log.gz 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40968.log 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46056.log: /var/log/ceph/ceph-client.admin.45924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47404.log 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46056.log.gz 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45924.log.gz 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43249.log 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40968.log.gz 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43446.log 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47404.log: /var/log/ceph/ceph-client.admin.43249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40802.log 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47404.log.gz 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43446.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43249.log.gz 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46881.log 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43446.log.gz 2024-09-12T01:35:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41454.log 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40802.log.gz 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47884.log 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47362.log 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41454.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46881.log.gz 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47884.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41454.log.gz 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45235.log 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41562.log 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47362.log: /var/log/ceph/ceph-client.admin.45235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49035.log 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47884.log.gz -- replaced with /var/log/ceph/ceph-client.admin.47362.log.gz 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45235.log.gz 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42907.log 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47800.log 2024-09-12T01:35:14.486 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41562.log.gz 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49035.log.gz 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48924.log 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42907.log.gz 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50611.log 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47800.log: /var/log/ceph/ceph-client.admin.48924.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47800.log.gz 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49161.log 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48924.log.gz 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46839.log 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50611.log: /var/log/ceph/ceph-client.admin.49161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49983.log 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50611.log.gz 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49161.log.gz 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47530.log 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46839.log.gz 2024-09-12T01:35:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48384.log 2024-09-12T01:35:14.488 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.49983.log.gz -5 2024-09-12T01:35:14.488 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43824.log 2024-09-12T01:35:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47530.log: /var/log/ceph/ceph-client.admin.48384.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47530.log.gz 2024-09-12T01:35:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48558.log 2024-09-12T01:35:14.488 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48384.log.gz 2024-09-12T01:35:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43246.log 2024-09-12T01:35:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43824.log: /var/log/ceph/ceph-client.admin.48558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46925.log 2024-09-12T01:35:14.488 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43824.log.gz 2024-09-12T01:35:14.488 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48558.log.gz 2024-09-12T01:35:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50445.log 2024-09-12T01:35:14.488 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43246.log.gz 2024-09-12T01:35:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-12T01:35:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46925.log: /var/log/ceph/ceph-client.admin.50445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45597.log 2024-09-12T01:35:14.489 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46925.log.gz 2024-09-12T01:35:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.audit.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50445.log.gz 2024-09-12T01:35:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46301.log 2024-09-12T01:35:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48210.log 2024-09-12T01:35:14.489 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45597.log.gz 2024-09-12T01:35:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41158.log 2024-09-12T01:35:14.489 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-12T01:35:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.48210.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46301.log.gz 2024-09-12T01:35:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40614.log 2024-09-12T01:35:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49203.log 2024-09-12T01:35:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41158.log: /var/log/ceph/ceph-client.admin.40614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50487.log 2024-09-12T01:35:14.489 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41158.log.gz 2024-09-12T01:35:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-12T01:35:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-12T01:35:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43636.log 2024-09-12T01:35:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36452.log 2024-09-12T01:35:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.3.43636.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-12T01:35:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-12T01:35:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36452.log: 91.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-12T01:35:14.502 INFO:teuthology.orchestra.run.smithi033.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.3.43636.log.gz 2024-09-12T01:35:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36332.log 2024-09-12T01:35:14.502 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36452.log.gz 2024-09-12T01:35:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-12T01:35:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36543.log 2024-09-12T01:35:14.503 INFO:teuthology.orchestra.run.smithi033.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36332.log.gz 2024-09-12T01:35:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36412.log 2024-09-12T01:35:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43187.log 2024-09-12T01:35:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36543.log: /var/log/ceph/ceph-client.admin.36412.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-12T01:35:14.503 INFO:teuthology.orchestra.run.smithi033.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36543.log.gz 2024-09-12T01:35:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.1.43187.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-12T01:35:14.503 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36412.log.gz 2024-09-12T01:35:14.503 INFO:teuthology.orchestra.run.smithi033.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-12T01:35:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-12T01:35:14.504 INFO:teuthology.orchestra.run.smithi033.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.1.43187.log.gz 2024-09-12T01:35:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-12T01:35:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-12T01:35:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36583.log 2024-09-12T01:35:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-mgr.x.log: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-12T01:35:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36372.log 2024-09-12T01:35:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36583.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36583.log.gz 2024-09-12T01:35:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-12T01:35:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36372.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36372.log.gz 2024-09-12T01:35:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36496.log 2024-09-12T01:35:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-mon.b.log: 92.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-12T01:35:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36292.log 2024-09-12T01:35:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36496.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36496.log.gz 2024-09-12T01:35:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36292.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36292.log.gz 2024-09-12T01:35:14.505 INFO:teuthology.orchestra.run.smithi033.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-12T01:35:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.49203.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43581.log 2024-09-12T01:35:14.555 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40614.log.gz 2024-09-12T01:35:14.555 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49203.log.gz 2024-09-12T01:35:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50235.log 2024-09-12T01:35:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50151.log 2024-09-12T01:35:14.555 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50487.log.gz 2024-09-12T01:35:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43581.log: /var/log/ceph/ceph-client.admin.50235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43247.log 2024-09-12T01:35:14.556 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43581.log.gz 2024-09-12T01:35:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50151.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50235.log.gz 2024-09-12T01:35:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45882.log 2024-09-12T01:35:14.556 INFO:teuthology.orchestra.run.smithi016.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.48210.log.gz 2024-09-12T01:35:14.556 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50151.log.gz 2024-09-12T01:35:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42606.log 2024-09-12T01:35:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40926.log 2024-09-12T01:35:14.556 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43247.log.gz 2024-09-12T01:35:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42606.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45882.log.gz 2024-09-12T01:35:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43866.log 2024-09-12T01:35:14.556 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42606.log.gz 2024-09-12T01:35:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42522.log 2024-09-12T01:35:14.557 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40926.log.gz 2024-09-12T01:35:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42438.log 2024-09-12T01:35:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43866.log: /var/log/ceph/ceph-client.admin.42522.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43866.log.gz 2024-09-12T01:35:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45998.log 2024-09-12T01:35:14.557 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42522.log.gz 2024-09-12T01:35:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43586.log 2024-09-12T01:35:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42438.log: /var/log/ceph/ceph-client.admin.45998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44597.log 2024-09-12T01:35:14.557 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42438.log.gz 2024-09-12T01:35:14.557 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45998.log.gz 2024-09-12T01:35:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46595.log 2024-09-12T01:35:14.557 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43586.log.gz 2024-09-12T01:35:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.44597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45429.log 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42648.log 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46595.log: 88.9% -- replaced with /var/log/ceph/ceph-client.0.44597.log.gz 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46967.log 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46595.log.gz 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45429.log.gz 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40884.log 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42648.log.gz 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46967.log: /var/log/ceph/ceph-client.admin.40884.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46967.log.gz -5 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49941.log 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40884.log.gz 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43782.log 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.49941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49390.log 2024-09-12T01:35:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43782.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49941.log.gz 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47488.log 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43782.log.gz 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48804.log 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49390.log.gz 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47274.log 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47488.log: /var/log/ceph/ceph-client.admin.48804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43238.log 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47488.log.gz 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48630.log 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48804.log.gz 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47274.log.gz 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49119.log 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43238.log.gz 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45277.log 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36463.log 2024-09-12T01:35:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49119.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48630.log.gz 2024-09-12T01:35:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45277.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.49119.log.gz 2024-09-12T01:35:14.560 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50025.log 2024-09-12T01:35:14.560 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45277.log.gz 2024-09-12T01:35:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50571.log 2024-09-12T01:35:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36463.log: /var/log/ceph/ceph-client.admin.50025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47179.log 2024-09-12T01:35:14.560 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36463.log.gz 2024-09-12T01:35:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50571.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50025.log.gz 2024-09-12T01:35:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41412.log 2024-09-12T01:35:14.560 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50571.log.gz 2024-09-12T01:35:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47179.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-12T01:35:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50653.log 2024-09-12T01:35:14.560 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47179.log.gz 2024-09-12T01:35:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41412.log: /var/log/ceph/ceph-mon.a.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.41412.log.gz 2024-09-12T01:35:14.561 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49773.log 2024-09-12T01:35:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45555.log 2024-09-12T01:35:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50653.log: /var/log/ceph/ceph-client.admin.49773.log: gzip -5 --verbose -- 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50653.log.gz 2024-09-12T01:35:14.561 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.43250.log 2024-09-12T01:35:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45555.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49773.log.gz 2024-09-12T01:35:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42690.log 2024-09-12T01:35:14.561 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45555.log.gz 2024-09-12T01:35:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43585.log 2024-09-12T01:35:14.561 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4%gzip/var/log/ceph/ceph-client.admin.42690.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47446.log 2024-09-12T01:35:14.561 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43250.log.gz 2024-09-12T01:35:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41202.log 2024-09-12T01:35:14.561 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42690.log.gz 2024-09-12T01:35:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47446.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43585.log.gz 2024-09-12T01:35:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-12T01:35:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47446.log.gz 2024-09-12T01:35:14.562 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose --/var/log/ceph/ceph-client.admin.41202.log: /var/log/ceph/ceph-client.admin.47950.log 2024-09-12T01:35:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41202.log.gz 2024-09-12T01:35:14.562 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.50193.log 2024-09-12T01:35:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36199.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47950.log.gz 2024-09-12T01:35:14.562 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-12T01:35:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44719.log 2024-09-12T01:35:14.562 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50193.log.gz 2024-09-12T01:35:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47656.log 2024-09-12T01:35:14.562 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36199.log.gz 2024-09-12T01:35:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41286.log 2024-09-12T01:35:14.562 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44719.log.gz 2024-09-12T01:35:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-12T01:35:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47656.log: /var/log/ceph/ceph-client.admin.41286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46637.log 2024-09-12T01:35:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mgr.y.log: 88.4% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47656.log.gz 2024-09-12T01:35:14.563 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41286.log.gz 2024-09-12T01:35:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42912.log 2024-09-12T01:35:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42312.log 2024-09-12T01:35:14.563 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46637.log.gz 2024-09-12T01:35:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42912.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-12T01:35:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42312.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42912.log.gz 2024-09-12T01:35:14.563 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42312.log.gz 2024-09-12T01:35:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49815.log 2024-09-12T01:35:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46259.log 2024-09-12T01:35:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49815.log: 85.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-12T01:35:14.564 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49815.log.gz 2024-09-12T01:35:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50403.log 2024-09-12T01:35:14.564 INFO:teuthology.orchestra.run.smithi016.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-12T01:35:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49731.log 2024-09-12T01:35:14.564 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46259.log.gz 2024-09-12T01:35:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47698.log 2024-09-12T01:35:14.564 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50403.log.gz 2024-09-12T01:35:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42564.log 2024-09-12T01:35:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49731.log: /var/log/ceph/ceph-client.admin.47698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46133.log 2024-09-12T01:35:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42564.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49731.log.gz 2024-09-12T01:35:14.564 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47698.log.gz 2024-09-12T01:35:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50529.log 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42564.log.gz 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42908.log 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46133.log: /var/log/ceph/ceph-client.admin.50529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41012.log 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42908.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46133.log.gz 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50529.log.gz 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42913.log 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42908.log.gz 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41984.log 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41012.log: /var/log/ceph/ceph-client.admin.42913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47093.log 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8%/var/log/ceph/ceph-client.admin.41984.log: -- replaced with /var/log/ceph/ceph-client.admin.41012.log.gz 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42913.log.gz 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50277.log 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48716.log 2024-09-12T01:35:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47093.log: /var/log/ceph/ceph-client.admin.50277.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47093.log.gz 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45471.log 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50277.log.gz 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.41984.log.gz 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46175.log 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48716.log: /var/log/ceph/ceph-client.admin.45471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42396.log 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48716.log.gz 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46175.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45471.log.gz 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42177.log 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48674.log 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42396.log: /var/log/ceph/ceph-client.admin.42177.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42396.log.gz 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46175.log.gz 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42177.log.gz 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44205.log 2024-09-12T01:35:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44076.log 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48674.log: /var/log/ceph/ceph-client.admin.44205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42235.log 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44076.log: 88.4% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48674.log.gz 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44205.log.gz 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41098.log 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41328.log 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44076.log.gz 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42235.log: gzip/var/log/ceph/ceph-client.admin.41098.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46427.log 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41328.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41098.log.gz 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42235.log.gz 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41328.log.gz 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43582.log 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42909.log 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46427.log.gz 2024-09-12T01:35:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43950.log 2024-09-12T01:35:14.568 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43582.log.gz 2024-09-12T01:35:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46797.log 2024-09-12T01:35:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42909.log: /var/log/ceph/ceph-client.admin.43950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47051.log 2024-09-12T01:35:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46797.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43950.log.gz 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42909.log.gz 2024-09-12T01:35:14.568 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-12T01:35:14.568 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46797.log.gz 2024-09-12T01:35:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46717.log 2024-09-12T01:35:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47223.log 2024-09-12T01:35:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47051.log: /var/log/ceph/ceph-client.admin.46717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45803.log 2024-09-12T01:35:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47223.log: 88.3% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47051.log.gz 2024-09-12T01:35:14.568 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46717.log.gz 2024-09-12T01:35:14.568 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47223.log.gz 2024-09-12T01:35:14.569 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47009.log 2024-09-12T01:35:14.569 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43580.log 2024-09-12T01:35:14.569 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45803.log: /var/log/ceph/ceph-client.admin.47009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47614.log 2024-09-12T01:35:14.569 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43580.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45803.log.gz 88.3% 2024-09-12T01:35:14.569 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47009.log.gz 2024-09-12T01:35:14.569 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43580.log.gz 2024-09-12T01:35:14.569 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36558.log 2024-09-12T01:35:14.569 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49432.log 2024-09-12T01:35:14.569 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47614.log.gz 2024-09-12T01:35:14.569 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43239.log 2024-09-12T01:35:14.569 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36558.log.gz 2024-09-12T01:35:14.569 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47316.log 2024-09-12T01:35:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49432.log: /var/log/ceph/ceph-client.admin.43239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50067.log 2024-09-12T01:35:14.570 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43239.log.gz/var/log/ceph/ceph-client.admin.47316.log: 2024-09-12T01:35:14.570 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49432.log.gz 2024-09-12T01:35:14.570 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47316.log.gz 2024-09-12T01:35:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-12T01:35:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49347.log 2024-09-12T01:35:14.570 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50067.log.gz 2024-09-12T01:35:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-12T01:35:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47135.log 2024-09-12T01:35:14.570 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49347.log.gz 2024-09-12T01:35:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45743.log 2024-09-12T01:35:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47135.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47135.log.gz 2024-09-12T01:35:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49857.log 2024-09-12T01:35:14.571 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45743.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45743.log.gz 2024-09-12T01:35:14.571 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43992.log 2024-09-12T01:35:14.571 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49857.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49857.log.gz 2024-09-12T01:35:14.571 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42354.log 2024-09-12T01:35:14.571 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43992.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43992.log.gz 2024-09-12T01:35:14.571 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49077.log 2024-09-12T01:35:14.571 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42354.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42354.log.gz 2024-09-12T01:35:14.571 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43584.log 2024-09-12T01:35:14.571 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49077.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49077.log.gz 2024-09-12T01:35:14.571 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43584.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43584.log.gz 2024-09-12T01:35:14.571 INFO:teuthology.orchestra.run.smithi016.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-12T01:35:14.571 INFO:teuthology.orchestra.run.smithi016.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-12T01:35:14.572 INFO:teuthology.orchestra.run.smithi033.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-12T01:35:14.579 INFO:teuthology.orchestra.run.smithi033.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-12T01:35:14.671 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-12T01:35:15.124 INFO:teuthology.orchestra.run.smithi033.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-12T01:35:15.142 INFO:teuthology.orchestra.run.smithi016.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-12T01:35:15.143 INFO:teuthology.orchestra.run.smithi033.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-12T01:35:15.153 INFO:teuthology.orchestra.run.smithi033.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-12T01:35:15.156 INFO:teuthology.orchestra.run.smithi016.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-12T01:35:15.195 INFO:teuthology.orchestra.run.smithi033.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-12T01:35:15.197 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-09-12T01:35:15.197 INFO:teuthology.orchestra.run.smithi033.stderr:real 0m0.730s 2024-09-12T01:35:15.197 INFO:teuthology.orchestra.run.smithi033.stderr:user 0m2.827s 2024-09-12T01:35:15.197 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m0.113s 2024-09-12T01:35:15.218 INFO:teuthology.orchestra.run.smithi016.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-12T01:35:15.284 INFO:teuthology.orchestra.run.smithi016.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-12T01:35:15.286 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-12T01:35:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m0.844s 2024-09-12T01:35:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m3.136s 2024-09-12T01:35:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m0.218s 2024-09-12T01:35:15.287 INFO:tasks.ceph:Archiving logs... 2024-09-12T01:35:15.287 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901553/remote/smithi016/log 2024-09-12T01:35:15.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T01:35:15.902 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901553/remote/smithi033/log 2024-09-12T01:35:15.903 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T01:35:16.303 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T01:35:16.313 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T01:35:16.313 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T01:35:16.316 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T01:35:16.421 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T01:35:16.430 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T01:35:16.430 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:35:16.433 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:35:16.449 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:35:16.451 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:35:16.453 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:35:16.454 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-12T01:35:16.454 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 -1818us[-1837us] +/- 56ms 2024-09-12T01:35:16.454 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 16 -294us[ -294us] +/- 54ms 2024-09-12T01:35:16.454 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +2512us[+2512us] +/- 65ms 2024-09-12T01:35:16.454 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:35:16.457 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:35:16.457 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-09-12T01:35:16.457 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 -1811us[-1792us] +/- 56ms 2024-09-12T01:35:16.457 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 -278us[ -278us] +/- 54ms 2024-09-12T01:35:16.457 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +2510us[+2510us] +/- 65ms 2024-09-12T01:35:16.457 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:35:16.458 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T01:35:16.467 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T01:35:16.467 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T01:35:16.477 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T01:35:16.574 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T01:35:16.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:35:16.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:35:16.678 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:35:16.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:35:16.710 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T01:35:16.720 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T01:35:16.740 INFO:teuthology.task.internal:Duration was 513.313194 seconds 2024-09-12T01:35:16.741 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T01:35:16.751 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T01:35:16.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T01:35:16.757 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T01:35:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:35:16.801 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:35:17.093 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T01:35:17.093 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:35:17.093 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T01:35:17.157 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-09-12T01:35:17.157 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T01:35:17.183 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T01:35:17.183 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T01:35:17.200 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T01:35:17.253 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T01:35:17.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T01:35:17.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T01:35:17.398 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T01:35:17.409 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T01:35:17.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T01:35:17.436 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T01:35:17.462 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T01:35:17.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:35:17.479 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:35:17.501 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-09-12T01:35:17.527 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-09-12T01:35:17.540 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:35:17.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:35:17.571 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:35:17.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:35:17.595 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T01:35:17.607 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T01:35:17.608 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901553/remote/smithi016 2024-09-12T01:35:17.608 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T01:35:17.645 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901553/remote/smithi033 2024-09-12T01:35:17.646 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T01:35:17.678 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T01:35:17.678 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T01:35:17.684 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T01:35:17.730 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T01:35:17.741 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T01:35:17.742 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T01:35:17.756 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T01:35:17.756 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T01:35:17.759 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T01:35:17.772 INFO:teuthology.orchestra.run.smithi016.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 01:35 /home/ubuntu/cephtest 2024-09-12T01:35:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 01:35 /home/ubuntu/cephtest 2024-09-12T01:35:17.787 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T01:35:17.796 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T01:35:17.867 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/cap-flush} duration: 513.3131937980652 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T01:35:17.867 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T01:35:17.970 INFO:teuthology.run:pass